changes in Tomcat 10 break Spring Boot 3 executable JARs

[Garret Wilson]

I'm passing on an issue discussed on Spring Boot GitHub which may not 
have been passed on to you. The main ticket is 
https://github.com/spring-projects/spring-boot/issues/33633 .


Spring Boot 3 executable JARs with Java 17 will break with 
"java.lang.IllegalStateException: zip file closed" when Tomcat 10 tries 
to start. You can see an example stack trace at 
https://github.com/spring-projects/spring-boot/issues/34028 .


Apparently Tomcat 9 used to do a JAR file multi-release check like this:

```java
@Override
public boolean jarFileIsMultiRelease(JarFile jarFile) {
    try {
    return ((Boolean) 
isMultiReleaseMethod.invoke(jarFile)).booleanValue();

    } catch (ReflectiveOperationException | IllegalArgumentException e) {
    return false;
    }
}
```

In Tomcat 10.1 it looks like this:

```java
multiRelease = jarFile.isMultiRelease();
```

I don't know anything about this code (I'm merely passing on what is in 
the ticket), but I can guess that Tomcat wants to get away from 
reflective operations, especially with later Java versions. That's fine 
and probably a good idea. Unfortunately the new code does no error 
handling whatsoever, so that the `IllegalStateException` kills 
everything. If you could at least catch the `IllegalStateException` and 
assume `false` I would imagine that would work around this issue.


The Spring Boot ticket acknowledges that they should improve their 
`JarUrlConnection` so that it wouldn't return a `JarFile` that was 
closed in the first place. Unfortunately there's no indication when that 
would be done, and right now the entire Spring Boot 3 executable JAR 
system is broken, requiring a system property to be added as a workaround.


Would you consider adding more error handling in a Tomcat 10 patch so 
that users to explicitly override the Tomcat 10 version and get their 
Spring Boot 3 executable JARs working again?


Thanks,

Garret


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Cannot find annotation method 'value()' in type 'aQute.bnd.annotation.spi.ServiceConsumer'

[Garret Wilson]

On 10/11/2022 9:18 AM, Mark Thomas wrote:

On 11/10/2022 16:36, Garret Wilson wrote:

…
  * The primary build generates JPMS and OSGI metadata, so some classes
    are annotated with the bnd annotation
    `aQute.bnd.annotation.spi.ServiceConsumer`. Currently this
    annotation come from `biz.aQute.bnd:biz.aQute.bnd.annotation:6.3.1`.


Not quite. Tomcat gets it from:

biz.aQute.bnd:biz.aQute.bnd:6.3.1


Perhaps that's where Tomcat gets it from. The library you mentioned 
seems to be some sort of an aggregate distribution of various libraries, 
perhaps to be used in an environment such as Ant that doesn't know how 
to go download transitive dependencies. The same class appears in the 
`biz.aQute.bnd:biz.aQute.bnd.annotation:6.3.1`.


…



  * The `aQute.bnd.annotation.spi.ServiceConsumer` annotation uses the
    OSGi annotation `org.osgi.annotation.bundle.Requirement`. Currently
    this annotation comes from `org.osgi:osgi.annotation:8.1.0`.


This is incorrect. bnd no longer depends on the OSGi JAR.


I believe what I said was correct: the 
`aQute.bnd.annotation.spi.ServiceConsumer` annotation uses the OSGi 
annotation `org.osgi.annotation.bundle.Requirement`. You can see this 
even in the `ServiceConsumer` source code for the 
`biz.aQute.bnd:biz.aQute.bnd:6.3.1` you mentioned:


```java
import org.osgi.annotation.bundle.Directive;
import org.osgi.annotation.bundle.Requirement;
…
@Requirement(name = VALUE_MACRO, namespace = SERVICELOADER_NAMESPACE, 
attribute = {
    SERVICELOADER_NAMESPACE + "=" + VALUE_MACRO, CARDINALITY_MACRO, 
RESOLUTION_MACRO

})
```

And even at 
https://search.maven.org/artifact/biz.aQute.bnd/biz.aQute.bnd/6.3.1/jar 
you can see that `biz.aQute.bnd:biz.aQute.bnd:6.3.1` declares 
dependencies on various OSGi artifacts. Perhaps in the primary Tomcat 
build, as Ant doesn't know how to pull down transitive dependencies, you 
never noticed that `biz.aQute.bnd:biz.aQute.bnd:6.3.1` relies on the 
OSGi libraries.


I'm not saying your build requires them; I'm simply saying that the 
source bnd source code uses them. If I include on the bnd libraries and 
not the OSGi libraries, `javac` tells me things such as:


> [WARNING] Cannot find annotation method 'value()' in type
> 'org.osgi.annotation.bundle.Requirements': class file for 
org.osgi.annotation.bundle.Requirements not found


I believe that the combination of dependencies I indicated includes the 
same classes required to prevent the warning without needing to include 
this larger "aggregate" artifact `biz.aQute.bnd:biz.aQute.bnd:6.3.1`.






  * Thus any secondary Maven builds need to inform Maven that it can
    download the bnd and OSGI annotations just to make sure that no
    classes are missing during this secondary build process, but that
    these artifacts are not needed at runtime and should not be
    distributed in the resulting JAR. In Maven this is easily indicated
    uses the `provided` scope.


This is not correct either. What you term secondary builds do not 
require bnd in order to compile against the Tomcat JARs.


I would quibble that I didn't say it was required; I said it was needed 
if you want to tell Maven that the classes aren't really missing and 
aren't needed at runtime.




This would not negatively affect Tomcat one bit, yet it would 
arguably provide "more complete information" for any secondary builds.


The negative impact is that all users of that JAR would be required to 
download the bnd library to compile their project for no benefit to them.


I understand your point of view, and I sincerely appreciate your 
spending valuable time in discussing this.


Cheers,

Garret


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Cannot find annotation method 'value()' in type 'aQute.bnd.annotation.spi.ServiceConsumer'

[Garret Wilson]

Mark, with your help I've figured this out and resolved the issue. Let 
me summarize and propose a more long-lasting solution. I don't expect 
you to agree, but please humor me and and least point out anything I 
understood incorrectly. (After all I need to go back and update the bug 
report. And the Stack Overflow question.)


 * Tomcat effectively has two builds:
1. The "primary build" uses Ant with `build.xml`, which compiles
   the source files, creates all the JARs and binaries, and
   publishes to Maven Central (Nexus).
2. Any "secondary build" by third parties using the published JARs
   and POMs using e.g.
   `org.apache.tomcat.embed:tomcat-embed-core:10.1.0` with Maven.
 * The latest versions of direct dependencies are found in the Tomcat
   repository inside `build.properties.default`.
 * The primary build generates JPMS and OSGI metadata, so some classes
   are annotated with the bnd annotation
   `aQute.bnd.annotation.spi.ServiceConsumer`. Currently this
   annotation come from `biz.aQute.bnd:biz.aQute.bnd.annotation:6.3.1`.
 * The `aQute.bnd.annotation.spi.ServiceConsumer` annotation uses the
   OSGi annotation `org.osgi.annotation.bundle.Requirement`. Currently
   this annotation comes from `org.osgi:osgi.annotation:8.1.0`.
 * The bnd and OSGi annotations remain part of the compiled classes
   even though they are not used at runtime and are not technically
   needed in any secondary builds.
 * Thus any secondary Maven builds need to inform Maven that it can
   download the bnd and OSGI annotations just to make sure that no
   classes are missing during this secondary build process, but that
   these artifacts are not needed at runtime and should not be
   distributed in the resulting JAR. In Maven this is easily indicated
   uses the `provided` scope.

Thus in a perfect world, Tomcat would simply indicate in its published 
POMs that these two dependencies contain necessary classes, but should 
not be distributed, using the following:


```xml

  biz.aQute.bnd
  biz.aQute.bnd.annotation
  6.3.1
  provided



  org.osgi
  osgi.annotation
  8.1.0
  provided

```

This would not negatively affect Tomcat one bit, yet it would arguably 
provide "more complete information" for any secondary builds.


Of course I can add those POM entries to my own secondary builds, and I 
will. But it's less than ideal. First of all a developer has to go track 
down exactly which dependencies are relevant. Secondly a developer has 
to dig inside `build.properties.default` to find out what versions are 
currently being used (and the versions of transitive dependencies are by 
no means obvious). Tomcat already has most of this information at 
primary build time, and could simply include it in the POM using the 
variable interpolation system it already uses.


Finally someone might ask, "Why do you care? It's a warning—just ignore 
it." Perhaps the person who asks this has never worked on a large 
project for a client and everything was breaking all over the place, and 
the build had 10,000 warnings and no one knew which of them actually 
were important and which were simply benign warnings that had been 
present for 15 years because of developer laziness. In any case, there 
are mechanics who work day to day with a messy tool shop and still just 
know where to find their tools; and other mechanics who like to keep 
their tool shop clean and organized. The latter type of mechanic might 
not be more or less competent, and it may not matter whether their tool 
shop is messy if they are working alone.


In today's world in which most software will be worked on by many 
developers across a multitude of dependencies, I prefer to keep a clean 
and tidy shop (i.e. project build). Thus I address any warnings, and if 
the warning is benign, I find a way to disable or suppress it. 
Suppressing a benign warning is best done nearest to where it is being 
generated, i.e. in this case within Tomcat itself, but since that isn't 
likely to happen I'll update my own build, which is an acceptable 
second-best option. At least now I know what's going on and how to 
address it.


Garret

On 10/11/2022 1:34 AM, Mark Thomas wrote:

On 10/10/2022 13:34, Garret Wilson wrote:

On 10/10/2022 12:14 AM, Mark Thomas wrote:

…

That is a fairly old version. You should upgrade to the latest 
version first.


I certainly will. I first went through a long list of linting 
problems in the project; this was one of the things left over. I 
wanted to understand it more.



If you want to fix the warning, those are not the correct dependencies.

If you want the exact dependencies used with 9.0.50 then you can 
find them from the source code:

…


You want
bnd (not bndlib) 5.3.0
OSGi annotations 1.1.0



That's super helpful. I'll go look for those.

But could someone give a 1–3 sentence overview of what these 
libraries are and why Tomcat uses them? Or is there a wiki page 
explaining

Re: Cannot find annotation method 'value()' in type 'aQute.bnd.annotation.spi.ServiceConsumer'

[Garret Wilson]

On 10/10/2022 12:14 AM, Mark Thomas wrote:

…

That is a fairly old version. You should upgrade to the latest version 
first.


I certainly will. I first went through a long list of linting problems 
in the project; this was one of the things left over. I wanted to 
understand it more.



If you want to fix the warning, those are not the correct dependencies.

If you want the exact dependencies used with 9.0.50 then you can find 
them from the source code:

…


You want
bnd (not bndlib) 5.3.0
OSGi annotations 1.1.0



That's super helpful. I'll go look for those.

But could someone give a 1–3 sentence overview of what these libraries 
are and why Tomcat uses them? Or is there a wiki page explaining that 
already?


Garret


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Cannot find annotation method 'value()' in type 'aQute.bnd.annotation.spi.ServiceConsumer'

[Garret Wilson]

I have a Java 11 project embedding Tomcat:

```xml

  org.apache.tomcat.embed
  tomcat-embed-core
  9.0.50

```

The Tomcat-specific code is in a subproject with only two classes. When 
I compile using Maven 3.8.6 and Java 17 using `-Xlint:all`, I see the 
following warning for that subproject:


> [WARNING] Cannot find annotation method 'value()' in type
> 'aQute.bnd.annotation.spi.ServiceConsumer':
> class file for aQute.bnd.annotation.spi.ServiceConsumer not found

As mentioned this only seems to happen with -`Xlint:all` turned on for 
`javac`:


```xml

  org.apache.maven.plugins
  maven-compiler-plugin
  
    true
    
  -Xlint:all
    
  

```

Doing a bit of searching brings up similar (but not exact) things, such 
as [Lombok Issue 
#2145](https://github.com/projectlombok/lombok/issues/2145), which hints 
that I may need to add some sort of extra dependency such as 
`biz.aQute.bnd:bndlib` or `org.osgi:osgi.annotation`.


But I've tried this and and I still get the warning:

```xml

  biz.aQute.bnd
  bndlib
  2.4.0

```

I also tried this; no difference:

```xml

  org.osgi
  osgi.annotation
  7.0.0

```

I also [asked on Stack Overflow](https://stackoverflow.com/q/74000505). 
No answers so far.


I filed [Bug 
66299](https://bz.apache.org/bugzilla/show_bug.cgi?id=66299] because I 
imagine that the Tomcat artifact isn't including some necessary 
dependency. That bug got closed because I'm supposed to ask here on this 
list first; nevertheless, including a Tomcat artifact should not result 
a warning regarding some other dependency I've never heard of and didn't 
refer to, so in any case this is incorrect behavior, and needs to be fixed.


Still I'd like to find out what's going on—maybe I can help you fix it. 
Where is this error coming from, and what does it mean? I don't have any 
`@ServiceConsumer` annotation in my source code, and I couldn't find any 
in the Tomcat classes I'm extending, either. Shouldn't the 
`org.apache.tomcat.embed:tomcat-embed-core` dependency contain 
everything I need (either in the artifact or via a transitive 
dependency) just to build a project that references the Tomcat classes? 
What is missing here?


Thanks,

Garret


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: completely automated (for real) Let's Encrypt on embedded Tomcat

[Garret Wilson]

On 10/7/2020 10:12 AM, Garret Wilson wrote:

…
But anyway, let me tell you the idea I had this morning. In a way, you 
hinted at it in your reply. Why do I need to use S3 as a store if my 
application is running on AWS, and AWS already has the AWS Certificate 
Manager which already manages an SSL certificate with renewal! In 
essence the AWS Certificate Manager is the "data store/state" like S3, 
and I don't even need to call Let's Encrypt.


Darn, it turns out AWS doesn't allow me to directly use AWS Certificate 
Manager certificates directly in my application.


   You cannot install your ACM certificate or your private ACM Private
   CA certificate directly on your AWS based website or application.
   (https://docs.aws.amazon.com/acm/latest/userguide/acm-services.html)

I guess it's back to Let's Encrypt using the original approach then.

I've already purchased a book on the intricacies of SSL certificates and 
I have more on my shopping list. I'll start working on this and get back 
to you with questions.


Cheers,

Garret

Re: completely automated (for real) Let's Encrypt on embedded Tomcat

[Garret Wilson]

As always thanks for the discussion, Chris. More replies and a new idea 
below:


On 10/6/2020 2:45 PM, Christopher Schultz wrote:

…
What if your Docker container would just run certbot on launch?


But then I'm back to being a sysadmin, because the Docker container is 
like a little OS and I have to set up the OS, update it, install certbot 
if needed (based on the OS version!), ensure it's the certbot that has 
the bugfixes I want and behaves as I expect, install Tomcat on the OS, 
etc. I have to set up certbot with systemd or or whatever to run 
periodically. All that stuff goes into my Dockerfile. So I'm really 
doing the same thing as I would on a VM, except that Docker makes it 
easier to reproduce. But it's conceptually not much different than being 
sysadmin on a VM, then freezing the VM and duplicating that.




If you build the keystore in-memory, I'm not exactly sure what you'd
need to do in order to get Tomcat to bounce the SSLSocketFactory in that
way.


OK, I'll look into it. But it boggles my mind that an SSLSocketFactory 
would have a harder time using actual bytes of a certificate than it 
would loading it from disk. Because once it loads it from disk, it has 
the bytes in memory, no? So the only problem would be the API—whether it 
was built to unnecessarily require disk storage, or if it allows a 
"hook" to provide the bytes at the point in the logic between 
loading-from-disk and using-the-cert.





I have no idea what a PEM-encoded DER file is, but I'll certainly learn.

This:

-BEGIN CERTIFICATE-
stuff
-END CERTIFICATE-

Ooh, that stuff. Yeah, I've definitely used that. I just never knew what 
it was. I never really got into the deep murky depths of certificates, 
but I guess now I'll have, seeing as that no one else in the last 20 
years has really made this easy.



I still don't get why files have to be involved. I pulled a DER file
from S3.

Sorry. I think of that as a "file".


Ah, there's an important distinction to be made here. As developers we 
often say "file" when we talk about a bunch of bytes that has some 
coherent format — a JPEG file, an Ogg Vorbis file, a JSON file. But (and 
the specs only really started making this clear around the time XML 
became a spec, as least as far as I know) from another view it's only a 
file if it's stored in a file system. (I'll explain why below.) So 
really we can talk about an "XML document", for example, that may never 
be stored in a file — we may construct an XML DOM tree completely in 
memory and pass it to some other method.




So your web server is S3.


Ah, no!

S3 is a database. It could be PostgreSQL or Cassandra. It's a data 
store. In particular it's a key-value store. It stores things. It has a 
feature that allows you to set a configuration so that some web server 
will automatically serve the BLOBs from the S3 data store, along with 
some metadata. But is the web server "part of" S3? I'll bet not. Surely 
there is some pool of workers that pull data from the S3 data store and 
serves the data. I could probably write my own web server to do the same 
thing. But AWS makes it transparent; it's part of the infrastucture. I 
just provide the data and declaratively tell it what I want served. Then 
it is served. I don't have to configure a server. There is some server 
in the cloud.


But there's actually another step! Since I'm serving the site via 
CloudFront, the CloudFront layer (servers around the world in different 
countries!) actually connects to the web site serving the S3 data and 
/copies it to CloudFront/! So my data is actually being served to the 
end user by some CloudFront server, in some country closest to the user. 
Is this Tomcat? Is it NGINX? Is it Apache? I don't know. Do I care what 
it is? No. In fact, I'd rather not know, because I want to configure the 
distribution and serving /independent from any server implementation/.


So my web server is something on some CloudFront deployment in some 
country. For all I know different CloudFront local deployments use 
different server types. I don't know and I don't care.




ELB ~= CloudFront, right?


Sort of. It is equivalent in that 1) it is the direct endpoint 
connection with the user, and 2) I can configure it to use the AWS 
managed certificates.




If CloudFront handles your SSL for you, why
not let ELB do it for you in this context?



That's what I'm doing now! And that's what I would prefer doing. And 
that's what I would need to do for a large application!


But the problem (going back to the motivation for all this) is that the 
ELB costs a lot of money. I don't absolutely need an ELB for a small app 
that I'm testing, or an app that will have 10 users, or an app that I 
don't care that crashes and immediately gets restarted. Why should I pay 
$16 or more per month for an ELB if I don't need it? When I have five 
little apps I want to deploy?


Think about the thousands of little apps people want to deploy around 
the world. Is 

Re: completely automated (for real) Let's Encrypt on embedded Tomcat

[Garret Wilson]

On 10/5/2020 2:42 PM, Christopher Schultz wrote:

…
Sure, it can contain S3 credentials and you can pick-up your key and
certificate (or, better yet, the whole keystore) there, but at that
point you have "moved" the problem outside of Tomcat, right?


No, not at all. The major problems are:

1. Generating the certificate automatically.
2. Feeding the certificate to Tomcat automatically.

If the "extra" thing I have to do is specify an environment variable 
with the name of the S3 bucket to use as a certificate state, then that 
is a teeny, tiny problem. That is not really even a problem.


Can you imagine if in my spring boot application I could run it using 
"java -jar my-app.jar --cert-work-bucket my-bucket" and it would just go 
get a certificate automatically?



You can have a "certificate renewal service" that writes to S3.


I want it built into my application as a module. You just include the 
module, specify the domain name (and maybe a couple of other details 
needed by RFC 8555, such as a contact email), and boom, it all happens 
automatically.


Why does it have to be more difficult than that? It shouldn't be.


I suppose you could put that directly into Tomcat, but Tomcat is not
likely to ship with an Amazon-specific feature built-into it.



Read my original email. I never intended to put this directly into 
standalone Tomcat (although if you want to put it into Tomcat you're 
welcome to). I want to use this with an application running on embedded 
Tomcat. Spring Boot is a prime example. I'll just extend the Spring Boot 
embedded Tomcat module and extend/modify that as needed. If that 
requires modifying Tomcat code, fine.



Another idea would be to use embedded Tomcat (or, at your suggestion,
Spring Boot) and fetch the keystore from some "standard" location of
your choosing. Again, that would be (appropriately, IMO) outside Tomcat
code.


That was always the intention. In my introduction to my original email I 
explained that the modern approach is moving away from something like 
standalone Tomcat to self-contained executables that run their own 
servers, whether embedded Tomcat or whatever. (I was late to the party, 
and even two years ago I wasn't getting it.)



1. Does acme4j allow me to verify my certificate behind another port?
    (e.g. ElasticBeanstalk deploys a JAR behind NGINX port 5000 by
    default. I'm still reading RFC 8555 to find out if the ACME server
    has to connect back on a certain port for verification.)
2. Once I have the Let's Encrypt certificate, can I convert to PKCS12
    for Tomcat completely in the application without shelling out to
    openssl or keytool? I'm hoping Bouncy Castle and/or acme4j-util will
    allow me to do that.
This can be done, but it's non-trivial. For example, Tomcat contains
code to package PEM-encoded DER files (good old OpenSSL-style =BEGIN
CERTIFICATE= things) into an in-memory keystore to configure JSSE.
It seems like it would be straightforward, but it turns out not to be in
all cases. YMMV.


Non-trivial as it may be, it /only needs to be done once/. If I have a 
converter, then I can use it a thousand times. A million times. And 
suddenly the deployment becomes a piece of cake.


In reality, today's style of handling SSL is what matches your 
description of "It seems like it would be straightforward, but it turns 
out not to be …". So why do we keep doing all this difficult, manual, 
tedious, not-trivial stuff to deploy certificates the hard way, when we 
could put our efforts into a single no-trivial task of making a 
converter so that Tomcat can use the Let's Encrypt certificates 
directly? We do it once. It's hard, but then everything else is easy. I 
don't get why we want to spend another decade doing it the hard way when 
we can spend one year on a different hard task and then do SSL the easy 
way for the other nine.


(The frustration isn't directed at you. It's just in general in software 
development I see the industry—why does the most basic of things have to 
be so difficult?)



3. Once I have the PKCS12, how do I feed it to the embedded Tomcat?
If it's a file on the disk, it's easy: just use the path.


Can I pass it in memory? If not, why not? How is memory less accessible 
than a file?



Chris, where can I get more information on the latter questions about
getting this certificate to Tomcat once I have it?

This mailing list is a good place to start (and likely finish).


Of course I really super-appreciate the help on this mailing list, and 
I'll be asking lots more questions. But I also don't want to ask things 
you've already answered elsewhere. I thought sure in one of your 
ApacheCon 2019 presentations you mentioned you had made more progress 
than the ApacheCon 2018 slides, such as auto-reloading or something. If 
there is further documentation let me know.



…
Go back to my presentation on Let's Encrypt and you'll see how to use
openssl to convert to a keystore if that's what you want.


I want 

Re: completely automated (for real) Let's Encrypt on embedded Tomcat

[Garret Wilson]

Thank you so much for replying, Chris. Responses below.

On 10/5/2020 8:53 AM, Christopher Schultz wrote:

Microservices won't work the way you want with Let's Encrypt. You have
two options:

1. Hit Let's Encrypt every time you launch a new instance of the
microservice to deploy a new certificate

2. Handle the certificate provisioning elsewhere (e.g. ELB)

#1 just won't work. LE won't re-issue a certificate more frequently than
every 6 weeks or something like that. So that really leaves you with #2.


It's good to know about the six-week limit, but you discarded #1 too 
quickly. Can't the microservice simply store the credentials in S3 or 
one of a hundred other data stores? (Note that I care less about 
"microservices" as such at the moment. I just want a turnkey deployment 
of a single application for now. But the idea is the same.)



What you really want is for the orchestrator to provide the certificate
and key to the nodes as they come on-line.


Look these "orchestrators" are a configuration nightmare at the moment. 
They end up being worse than my just configuring a CentOS machine from 
scratch. Plus I have to pay for all those extra services. Read 
https://leebriggs.co.uk/blog/2019/04/13/the-fargate-illusion.html and 
try not to shudder.



So instead of trying to get LE to work with Tomcat (which does work, but
requires some care and feeding), maybe we should try to get Tomcat to
load its crypto material from other places.


There is already a Java library (https://github.com/shred/acme4j) for 
talking to Let's Encrypt. It sounds like it does everything I need. I'll 
need to investigate more, but here are my initial doubts:


1. Does acme4j allow me to verify my certificate behind another port?
   (e.g. ElasticBeanstalk deploys a JAR behind NGINX port 5000 by
   default. I'm still reading RFC 8555 to find out if the ACME server
   has to connect back on a certain port for verification.)
2. Once I have the Let's Encrypt certificate, can I convert to PKCS12
   for Tomcat completely in the application without shelling out to
   openssl or keytool? I'm hoping Bouncy Castle and/or acme4j-util will
   allow me to do that.
3. Once I have the PKCS12, how do I feed it to the embedded Tomcat?

Chris, where can I get more information on the latter questions about 
getting this certificate to Tomcat once I have it?



One way to do that would be with e.g. Amazon's key storage service. I'm
not familiar with that. I know it can store various types of keys; not
sure about certificates and if we can pull a private key out of it.


I think your idea of storing this stuff elsewhere is a good first 
stepping stone to get to where I want to go.


Just to get the ball rolling, I could manually run some Let's Encrypt 
client, and then store the certificate in S3. Then the first component I 
would write would be steps #2 and #3 above. I am already familiar with 
the AWS Java library, so I could quickly figure out how to pull the 
certificates off S3. But I need your help in finding out how to convert 
them to PKCS12 (or whatever; this is new territory for me) on the fly 
and feed them to the embedded Tomcat.



Honestly, your best bet would probably be to use ELB and just pay for
it. You only pay for data-transfer, so a dormant ELB costs you virtually
nothing.


Last month I deployed a test application on Elastic Beanstalk on a 
domain nobody knows about just to see how it worked. The ELB cost me $16 
in a month with basically nobody using it! That adds up quickly. I have 
several little apps I want to toss up. See also my question 
https://serverfault.com/q/1036276 .




Instead of spinning-up an EC2 instance for your service, maybe you
should be looking at Lambda instead. You can probably get your costs
down more that way than trying to eliminate the ELB.


I want to drop either a self-contained JAR file or a Docker image 
somewhere, and have it immediately start running with SSL support, 
without my configuring a VM or running scripts. When I have a new 
version, I want to drop a new JAR or Docker image and have it 
automatically replace the other one. I don't want to maintain a VM. I 
have a target price of let's say $5/month for everything (although $3 
would be better). Does AWS Lambda give me that? If so, please point me 
to the guides.


Garret

completely automated (for real) Let's Encrypt on embedded Tomcat

[Garret Wilson]

Hi, everyone. I'm back already. (I had intended to leave the list to 
focus my efforts elsewhere, but … here I am again.)


I just realized there is a big SSL problem for small applications, and I 
want to fix it. First a little review of where we are.


Servlet containers are becoming less important and less desirable in 
today's world, because we don't want to deploy and maintain some sort of 
high-level container infrastructure (in the Java EE container sense, not 
the Docker sense) just to deploy an application in it. Modern 
distributed micrososervice applications have a bunch of 
service/worker/agent application that are identical and redundant. You 
spin up as many as you need; if some go down, you (or an orchestrator) 
spins up others.


For this reason libraries like Spring Boot allow you to deploy your Java 
application as a standalone JAR with embedded Tomcat. The JAR represents 
the completely independent application. You just throw it on a node and 
it runs and provides a web server or whatever. So we we should be able 
to throw a Spring Boot JAR on something like AWS Elastic Beanstalk and 
it just runs. I found out it is far from that simple, and SSL is one of 
the major problems.


There seem to be two ways to get SSL support. On something like AWS 
Elastic Beanstalk, you deploy a load balancer in front of your EC 
instances. Elastic Beanstalk will (using the AWS Route 53 DNS) configure 
SSL to the load balancer, spin up EC instances as needed (each running 
your standalone JAR), and connect the load balancer to the EC instances, 
all in a (sort of) automated fashion. But note that the SSL endpoint is 
the load balancer, and the load balancer costs money! Even if you're 
just running just a single standalone JAR instance requiring a single EC 
instance, that load balancer sits there and drains cash. Significant 
cash if you just want to run a little program with SSL support.


What's the other option to deploy a standalone JAR? Configure an AWS EC 
instance (or a VM with another provider), configure certbot, configure 
Tomcat, save some files locally on the machine, etc. All this manual 
work. I just want to run the standalone JAR! In short, if I have a 
standalone program I want to run, I either have to configure and 
maintain a VM like I did in the year 2000, or get into the nightmare of 
Kubernetes-like orchestration with the endless configurations and/or the 
high costs.


I propose to create a module that integrates with embedded Tomcat that:

1. You indicate what domain you're hosting for (as part of the
   application configuration or as an environment variable when
   deployed, for example).
2. When your application starts running, it automatically connects to
   Let's Encrypt using RFC 8555 (or whatever is needed) and requests a
   certificate, based upon the IP address it's running on.
3. The module exposes the correct HTTP paths and/or connects to a
   configured DNS as needed for validation.
4. The module receives the certificates and caches them in memory or in
   a temporary file as needed and provides them to Tomcat; Tomcat now
   is serving using SSL/TLS.
5. If the application dies, who cares? You start up another one. It
   automatically does the same thing (on another machine or wherever it
   is running) and the application is running SSL/TLS. It's that
   simple. You don't need to run certbot. You don't need to manually
   copy files on the system. You don't even need to know where the
   application is going to run. You just need an executable JAR with
   this new module, and you run it. Done.
6. (Many variations exists where multiple JARs are running but one is
   the "leader" for Let's Encrypt, and they communicate and share the
   cashed certificate until the node dies. Or there are variations
   using Docker. The first step is the radical one, and then all sorts
   of possibilities open up.)

From glancing over the Let's Encrypt docs and having had hands-on 
experience embedding Tomcat, that seems completely doable to me. And I'm 
ready to start.


But first, what work has been done in this area already? I'm aware of 
Chris' slides from 2018, but those techniques require some combination 
of certbot, keytool, non-embedded Tomcat, symlinks,OS scripts, manually 
file system manipulation, etc. I think at ApacheCon 2019 Chris mentioned 
some more work has been done on this, but I don't recall where it was.


Please point me to the latest work and ideas for Tomcat+Let's Encrypt so 
that I don't spend two months doing something that is already been done, 
or before I find out it is impossible.


As it stands I want fully automated SSL/TLS configuration just by 
running a standalone JAR, and I don't see that existing anywhere. I'm 
not prepared to pay AWS for a load balancer just to run a little app, 
and I got tired of manual Linux setup and scripts and general sysadmin 
work around 20 years ago. It's the cloud. It should act like the cloud.


Garret

Re: using custom classes in Tomcat configuration files

[Garret Wilson]

Hi everyone. I thought I would circle back and answer my own question 
from back in June 2020 (see message thread below).


The thing that creates objects from the server.xml configuration file is 
the Digester 
(https://tomcat.apache.org/tomcat-9.0-doc/api/org/apache/tomcat/util/digester/Digester.html), 
and it uses the processing rules described in the digester package 
summary 
(https://tomcat.apache.org/tomcat-9.0-doc/api/org/apache/tomcat/util/digester/package-summary.html#doc.Rules).


Of particular note are the ObjectCreateRule 
(https://tomcat.apache.org/tomcat-9.0-doc/api/org/apache/tomcat/util/digester/ObjectCreateRule.html) 
and the SetPropertiesRule 
(https://tomcat.apache.org/tomcat-9.0-doc/api/org/apache/tomcat/util/digester/SetPropertiesRule.html).


You can read more about my upgrading my Guise Mummy components to be 
able to be used in Tomcat configuration files, as well as see related 
code changes, at https://globalmentor.atlassian.net/browse/GUISE-150 .


Now that I have all this working, I won't be coding on Tomcat as often, 
so I'm going to drop off this list for a while to cut down on my inbox 
influx. Thanks for the opportunity to participate in ApacheCon last 
year, and I hope the virtual event this year is a success. I enjoyed 
meeting those of you I met in person, and if things go well maybe I'll 
meet some of you again at an in-face conference in the future.


All the best and stay safe.

Garret

On 6/8/2020 6:59 AM, Garret Wilson wrote:
Hi, all. I'm getting close to releasing a new version of my Guise 
Mummy static site generation software.


I gave a presentation over Guise Mummy's use of embedded Tomcat to 
serve the generated site locally for testing before deployment. It 
uses a custom site root and web resource


https://guise.io/mummy/present/clean-urls

Although in my use case I instantiation and connect these things 
programmatically, someone in attendance (I think it was Remy, or maybe 
Chris) mentioned that I could also reference the customized versions 
in one of the configuration files and Tomcat would create the instance 
graph automatically.


I want to make the library flexible, so in this version I'd like to 
make sure that's possible. Could you remind me on the prerequisites 
for this, and what documentation I could look at? Does it use JavaBean 
conventions or some other conventions? (I assume I just need a 
no-argument constructor and some getters/setters, but I want to 
verify.) And in which Tomcat configuration would they go?


Thanks,

Garret


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

using custom classes in Tomcat configuration files

[Garret Wilson]

Hi, all. I'm getting close to releasing a new version of my Guise Mummy 
static site generation software.


I gave a presentation over Guise Mummy's use of embedded Tomcat to serve 
the generated site locally for testing before deployment. It uses a 
custom site root and web resource


https://guise.io/mummy/present/clean-urls

Although in my use case I instantiation and connect these things 
programmatically, someone in attendance (I think it was Remy, or maybe 
Chris) mentioned that I could also reference the customized versions in 
one of the configuration files and Tomcat would create the instance 
graph automatically.


I want to make the library flexible, so in this version I'd like to make 
sure that's possible. Could you remind me on the prerequisites for this, 
and what documentation I could look at? Does it use JavaBean conventions 
or some other conventions? (I assume I just need a no-argument 
constructor and some getters/setters, but I want to verify.) And in 
which Tomcat configuration would they go?


Thanks,

Garret


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: [ANN] Apache Tomcat 10.0.0-M5 available

[Garret Wilson]

Thanks for the announcement.

Is there any rough timeline or roadmap for a stable and/or release 
version of Tomcat 10? (Sorry if this has been discussed here already.)


I'm in no rush. I just have an application with embedded Tomcat which is 
due for another release soon, and I wondered whether it was time to 
start looking into switching yet. Is Tomcat 10 pretty solid and the API 
stable for simple embedded serving?


Thanks,

Garret

On 5/12/2020 7:11 AM, Mark Thomas wrote:

The Apache Tomcat team announces the immediate availability of Apache
Tomcat 10.0.0-M5.

Apache Tomcat 10 is an open source software implementation of the
Jakarta Servlet, Jakarta Server Pages, Jakarta Expression Language,
Jakarta WebSocket, Jakarta Authentication and Jakarta Annotations
specifications.

Users of Tomcat 10 onwards should be aware that, as a result of the move
from Java EE to Jakarta EE as part of the transfer of Java EE to the
Eclipse Foundation, the primary package for all implemented APIs has
changed from javax.* to jakarta.*. This will almost certainly require
code changes to enable applications to migrate from Tomcat 9 and earlier
to Tomcat 10 and later. A migration tool is under development to aid
this process.

Apache Tomcat 10.0.0-M5 is a milestone release of the 10.0.x
branch and has been made to provide users with early access to the new
features in Apache Tomcat 10.0.x so that they may provide feedback. The
notable changes compared to 10.0.0-M4 include:

- Remove useAprConnector flag from AprLifecycleListener so that the
   only way to use the APR connectors is to set the full class name.

- Change default value separator for property replacement to ":-"
   due to possible conflicts. The syntax is now "${name:-default}".

- Update the packaged version of the Tomcat Native Library to 1.2.24.

Please refer to the change log for the complete list of changes:
http://tomcat.apache.org/tomcat-10.0-doc/changelog.html

Downloads:
http://tomcat.apache.org/download-10.cgi

Migration guides from Apache Tomcat 7.0.x, 8.5.x and 9.0.x:
http://tomcat.apache.org/migration.html

Enjoy!

- The Apache Tomcat team

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: [OT] distinction between resource charset and format octet decoding

[Garret Wilson]

On 2/6/2020 12:43 PM, Christopher Schultz wrote:

…

* Therefore `web.xml` settings, HTTP headers, etc. are all
irrelevant, as this is an issue dealing with the file format
itself, and the latest spec for the file format says to use UTF-8,
so everyone should use UTF-8 already.

Except for everyone who already uses something else and expects
everything to be backward-compatible.


I think there comes a time where we have to more forward after some 
critical level of usage is reached. I think we've passed that point.


Modern browsers in the sense that you mention are not 
backwards-compatible for `application/x-www-form-urlencoded`. So what 
are we being compatible with by not using UTF-8 decoding? Do we have 
anything besides browsers consuming output from legacy JSP apps? As 
noted the browsers break when we try to be "backwards-compatible" in the 
sense you mention.



The problem is that you don't get to declare what's "best" for
everyone and then the whole world does what you want.


But here I would imagine that already agrees what's best; the debate is 
whether we should do different than what we know is best because of some 
outdated specs. (And I say that as a huge proponent of following standards.)


I'll give you an example that is directly relevant. Over 10 years ago I 
strongly advocated to the RDF group that the Internet should abandon the 
outdated practice of requiring that `text/*` media types default to 
US-ASCII; otherwise there would be no point in using `text/*` for 
anything going forward! (That's why we went through a sad phase where 
everyone was using `application/*` for text formats because they wanted 
to default to something other than US-ASCII.)


 * https://www.w3.org/2008/01/rdf-media-types
 * https://lists.w3.org/Archives/Public/www-archive/2007Dec/0059.html

Sure enough, eventually someone saw the light (I won't claim I had 
anything to do with it, but it is exactly what I was arguing for) and 
created https://tools.ietf.org/html/rfc6657, which says that individual 
`text/*` types can choose a default other than ASCII. Finally we're not 
stuck in the past anymore!


I would say that someone needs to create an updated 
`application/x-www-form-urlencoded` specification prescribing UTF-8 
decoding of encoded octets, except that the WhatWG has already done 
that! So I'm not declaring that everyone should do it "my" way. I'm 
saying everyone should follow the latest spec which already exists.


Anyway, thanks for listening. I think it's a fun discussion, and I 
wasn't being combative---I just wanted to tell a bit of the story. I 
need to get back to work now. :)


Thanks again for the change in Tomcat 10!

Garret

Re: [OT] distinction between resource charset and format octet decoding

[Garret Wilson]

On 2/6/2020 11:46 AM, André Warnier (tomcat/perl) wrote:

…

As of Tomcat 10, conf/web.xml contains the following:


UTF-8
UTF-8

That *should* have the effect you are looking for but I confess I
haven't tested it in any great detail.



As I am sure many people (Christopher included) would agree, the real 
solution would be for browsers and other HTTP clients to indicate 
clearly in the request, the charset/encoding of each text parameter 
that they are sending.

There are even HTTP headers already defined for that.



Which HTTP headers are you referring to? `Content-Type`? It is my 
opinion that this is irrelevant and not applicable.


As I explained (extensively) in my original post for this thread back on 
2019-01-08, the issue is not the charset of 
`application/x-www-form-urlencoded`. That media type is made up of ASCII 
characters. It doesn't matter whether you say it's ASCII, ISO-8859-1, 
UTF-8, or whatever, the actual characters stay 100% the same. At issue 
is when certain octets are encoded (as specified by the 
`application/x-www-form-urlencoded` media type itself), what charset to 
use when decoding them. This is independent of the encoding of the media 
type itself; rather this is defined by the specification for the format.


Unfortunately https://tools.ietf.org/html/rfc1866 actually says we 
should use ASCII when decoding the octets, but this is severely 
antiquated and doesn't fit with modern practice. The WhatWG essentially 
redefines the format to say that the octets must be interpreted as UTF-8:


https://url.spec.whatwg.org/#application/x-www-form-urlencoded

So to summarize my view:

 * The decoding of the `application/x-www-form-urlencoded` media type
   encoded octets is completely independent of the charset indicated in
   the `Content-Type` header, and rather goes to the specification of
   the format itself.
 * RFC 1866 is severely out of date and out of step, and the WhatWG's
   specification of the `application/x-www-form-urlencoded` media type
   should be used instead. (Modern browser practice would seem to agree
   with me.)
 * Therefore `web.xml` settings, HTTP headers, etc. are all irrelevant,
   as this is an issue dealing with the file format itself, and the
   latest spec for the file format says to use UTF-8, so everyone
   should use UTF-8 already.

The new default `web.xml` in Tomcat 10 is a wonderful step in the right 
direction.


See my original post for more in-depth explanation.

Garret

Re: distinction between resource charset and format octet decoding

[Garret Wilson]

On 2/6/2020 10:44 AM, Mark Thomas wrote:

…
As of Tomcat 10, conf/web.xml contains the following:


UTF-8
UTF-8

That *should* have the effect you are looking for but I confess I
haven't tested it in any great detail.


Yes! Oh, that is so wonderful. Thank you!

I brought this issue up on the list over a year ago, and I have since 
published my entire comprehensive software development course (still 
being expanded).


https://www.globalmentor.com/courses/softdev/

The course is centered around Tomcat as the server, and the lesson on 
HTML forms contains a section warning to use ``.


https://www.globalmentor.com/courses/softdev/html-forms

Once Tomcat 10 is released I'll be able to update this note as well.

Thanks again!

Garret


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: distinction between resource charset and format octet decoding

[Garret Wilson]

On 2/6/2020 10:36 AM, Mark Thomas wrote:

…

Whether Tomcat should ship with this setting present in conf/web.xml
by default is something that should probably be discussed for Tomcat
10. Given the current state of the web, there is a reasonable case for
doing so. I'll add that to the TOMCAT-NEXT discussion list.

Is this still on the list for discussion for Tomcat 10?

No, because it has already been implemented for Tomcat 10 and is in the
milestone release currently being voted on.


Waitasec. I'm not used to good news, so I want to make sure I understand 
what you're saying. Are you saying that the proposed Tomcat 10 
implementation already interprets encoded octets in web form submissions 
using UTF-8 by default?!! :O


It will be a joy to update the FAQ when this is released.

Garret


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: distinction between resource charset and format octet decoding

[Garret Wilson]

On 1/8/2019 9:57 PM, Mark Thomas wrote:

…

Yes, this default is now very out-dated. That is a side-effect of:
…
As of Servlet 4.0 there is a specification compliant configuration 
option to change this default to any encoding of your choice. 
Obviously, UTF-8 is one of the options. You can do this by adding the 
following to your web.xml:

…

Whether Tomcat should ship with this setting present in conf/web.xml 
by default is something that should probably be discussed for Tomcat 
10. Given the current state of the web, there is a reasonable case for 
doing so. I'll add that to the TOMCAT-NEXT discussion list.


Is this still on the list for discussion for Tomcat 10?

In my opinion it would be a real shame if Tomcat 10 ships with a web 
form encoding default that goes against the WhatWG specifications and 
corrupts non ISO-8859-1 content under modern browsers.


Garret

P.S. Mark, please ignore the other email from my personal email address. 
Because the Tomcat users list doesn't include my name in the "To:" 
header, my email client didn't know to use the correct reply address.



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: efficient redirect map with embedded Tomcat

[Garret Wilson]

On 10/15/2019 6:06 AM, Christopher Schultz wrote:



(The use case is simply to migrate some old URLs that have probably
been indexed already or even linked on the web. Theoretically the
entire site would need to redirect its old URLs, but probably only
the pages.)

So, just to be sure, you are talking about HTTP 302 "redirect"
responses, and not just mapping incoming URIs to some other resource,
right?


Actually HTTP 301, because these represent resources that have 
permanently moved to the new location.


But yes, in essence that's what we're talking about: when a request 
comes in for an old path, we send back an HTTP redirect to the new path. 
No content will be served at the old path.


Garret


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: efficient redirect map with embedded Tomcat

[Garret Wilson]

On 10/13/2019 11:52 AM, Mark Thomas wrote:

That depends on how you define best. Simplest to implement? Easiest to
maintain? Minimum overhead?


How about, "What best follows the spirit of the Tomcat architecture?"

Or alternatively, "What would be most efficient (i.e. not slowing down 
normal requests)?"




It also depends on how many redirects are you talking about as well as
what sort of % of the over all requests need to be redirected.


Let's say 100 resources need redirecting, to pick an arbitrary number.

(The use case is simply to migrate some old URLs that have probably been 
indexed already or even linked on the web. Theoretically the entire site 
would need to redirect its old URLs, but probably only the pages.)


Garret


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: efficient redirect map with embedded Tomcat

[Garret Wilson]

Could somebody at least point me to the best place to wire in site-level 
per-resource redirects in embedded Tomcat? I can create a solution, I 
just need to know where it is best to start.


Thanks,

Garret

On 10/11/2019 11:06 AM, Garret Wilson wrote:
This is a question for Tomcat experts before I get started 
implementing a new feature.


Let's say I'm embedding Tomcat to serve static files. At the time of 
creation I know that certain paths, such as `foo/bar.txt`, should 
redirect to other paths, such as `some/other.txt`. What's the best way 
to configure Tomcat to do those redirects? I'm comfortable with 
extending the source code.


Here are a couple of ideas that come to mind:

 * I could create a redirect servlet and map different instances of it
   to different targets in the context when I configure everything. But
   in Tomcat's routing engine, is the most efficient way to do things?
   (I assume that the servlet mappings can be placed "over" the default
   servlet's path space, that is, cherry-pick paths for redirection,
   falling back to the default file-serving servlet for non-redirect
   paths.)
 * I thought of patching into the default file servlet, overriding
   `org.apache.catalina.WebResource`, and creating virtual
   `RedirectResource` resources that don't correspond to any physical
   file. However it's not obvious to me where I would create a
   redirect. Maybe throw some redirect exception inside
   `WebResource.getInputStream()`? (This is probably not correct. I'm
   just brainstorming. The idea is sound if I knew where to put it.)
 * Should I install a configured rewrite valve when I'm setting up
   embedded Tomcat?
 * Is there some other routing logic in Tomcat I could tap into most
   efficiently, providing a known set of redirects?

Thanks for any guidance. I'm want to figure out the best way to attack 
this before getting very deep in an implementation.


Garret




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

efficient redirect map with embedded Tomcat

[Garret Wilson]

This is a question for Tomcat experts before I get started implementing 
a new feature.


Let's say I'm embedding Tomcat to serve static files. At the time of 
creation I know that certain paths, such as `foo/bar.txt`, should 
redirect to other paths, such as `some/other.txt`. What's the best way 
to configure Tomcat to do those redirects? I'm comfortable with 
extending the source code.


Here are a couple of ideas that come to mind:

 * I could create a redirect servlet and map different instances of it
   to different targets in the context when I configure everything. But
   in Tomcat's routing engine, is the most efficient way to do things?
   (I assume that the servlet mappings can be placed "over" the default
   servlet's path space, that is, cherry-pick paths for redirection,
   falling back to the default file-serving servlet for non-redirect
   paths.)
 * I thought of patching into the default file servlet, overriding
   `org.apache.catalina.WebResource`, and creating virtual
   `RedirectResource` resources that don't correspond to any physical
   file. However it's not obvious to me where I would create a
   redirect. Maybe throw some redirect exception inside
   `WebResource.getInputStream()`? (This is probably not correct. I'm
   just brainstorming. The idea is sound if I knew where to put it.)
 * Should I install a configured rewrite valve when I'm setting up
   embedded Tomcat?
 * Is there some other routing logic in Tomcat I could tap into most
   efficiently, providing a known set of redirects?

Thanks for any guidance. I'm want to figure out the best way to attack 
this before getting very deep in an implementation.


Garret

Re: ApacheCon NA Tomcat track videos

[Garret Wilson]

For the equipment setup, these videos are pretty good. Thanks for 
providing these, Rémy.


Garret

On 9/18/2019 1:11 PM, Rémy Maucherat wrote:

Hi,

I finished uploading the videos of the twelve sessions from the Apache
Tomcat track at  ApacheCon NA 2019 in Las Vegas to the Apache Tomcat
YouTube channel:
https://www.youtube.com/playlist?list=PLI91YOalv6p9bivF8eb_MR9bVQcVfLkH9

The sound recording quality is rather bad, apologies for that. I'll buy
basic movie-recording-with-smartphone equipment in case I have to do it
again, but the whole thing was improvised this time.

Enjoy ! [hopefully ;) ]

Rémy



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: distinction between resource charset and format octet decoding

[Garret Wilson]

Sorry to bring up the non-UTF-8 escaped octets form POST problem again, 
but …


On 1/8/2019 3:57 PM, Mark Thomas wrote:

…
As of Servlet 4.0 there is a specification compliant configuration 
option to change this default to any encoding of your choice. 
Obviously, UTF-8 is one of the options. You can do this by adding the 
following to your web.xml:


UTF-8

If you add it to conf/web.xml it applies to every web application 
deployed to Tomcat.


Tomcat 9 uses this in the examples, manager and host-manager 
applications in place of the SetCharacterEncodingFilter.



As you know I've already updated the Tomcat FAQ with the options for 
forcing Tomcat to interpret form POSTs with any escaped characters using 
UTF-8 octet sequences (as modern browsers send, and as HTML5 requires) 
instead of ISO-8859-1 (as the Servlet 4 spec says).


But the problem is worse with the Spring community. If someone is using 
Spring Boot to create an executable JAR/WAR using embedded tomcat, 
Spring Boot does something to configure Tomcat to send the POSTs 
correctly (that is, as the modern web likes it, not like the Servlet 4 
spec says). Unfortunately, if I use Spring Boot to make a WAR which is 
both a self-contained executing WAR /and/ a WAR deployable on Tomcat, 
when I deploy the WAR on Tomcat the encoded characters are using escaped 
ISO-8859-1 octets, so my web app breaks. Yes, the WAR runs differently 
if using Spring Boot embedded Tomcat or deployed on standalone Tomcat as 
a WAR.


Spring Boot ignores any `web.xml` file. I guess I could create a 
`web.xml` file only for standalone Tomcat, but then this freezes Eclipse 
(as I posted elsewhere) because Eclipse doesn't understand 
``. So like so many things on the web, this 
is a mess.


This is a serious issue, in my opinion. The Servlet 4 specification is 
out of step with everything else in the ecosystem!


Whether Tomcat should ship with this setting present in conf/web.xml 
by default is something that should probably be discussed for Tomcat 
10. Given the current state of the web, there is a reasonable case for 
doing so. I'll add that to the TOMCAT-NEXT discussion list.


Yes, can I just re-second (third?) that motion, and underscore the need 
for this to be changed in Tomcat 10?


Thanks,

Garret

Re: small extension for Tomcat in ApacheConNA2019

[Garret Wilson]

On 5/14/2019 5:27 AM, Christopher Schultz wrote:

…

Is that the sort of of level of talk they are looking for, and is
it of appropriate general interest for the conference?

Your proposal will be put into a pool of source material for the
conference and it will be weighed against the other submissions. So
let the organizers decide whether or not the content is the right
material.

In the past two years, we have had a lot of presentations coming from
"the community" which have generally been along the lines of what you
have above, rather than "here's how to configure product X". I tend to
prefer presentations that come from outside the core committers,
because it shows what you guys are actually DOING with e.g. Tomcat.

I'd encourage everyone to submit anything they think might be even
somewhat related. You don't have to write the presentation before
proposing it. Most presenters I know of are often continuing to work
on their slides up until the day of their presentations :)


Thanks, Chris! I was just wanted to confirm I was in the general 
vicinity of subject topics.


I went ahead and submitted the proposal.

Cheers,

Garret


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: small extension for Tomcat in ApacheConNA2019

[Garret Wilson]

Hi, Jean-Frederic. Could you give an indication of what sort of 
presentations they are looking for? Would a presentation of an 
open-source Tomcat extension, with architecture/code explanation and 
live demonstration, be appropriate? For example:


   *Title:* Serving Clean Resource Names with Custom MIME Internet
   Types on Tomcat

   *Summary:* Increasingly static site generators are being used to
   create a presence on the web, both for the speed and simplicity of
   the resulting static pages. Coupled with this trend is the use of
   "clean" resource names with no extensions, such as
   `/products/mousetrap` instead of `/products/mousetrap.html`. Because
   the default Tomcat implementation determines MIME type by filename
   extension, "clean" filenames are traditionally produced by URL
   redirects, perhaps using Apache HTTP Server in conjunction with
   `mod_rewrite` in front of Tomcat. This presentation shows how Tomcat
   has been extended to dynamically determine MIME type at runtime
   based on metadata stored in a parallel file tree or as sidecar
   files, and to serve the pages directly from Tomcat with no need for
   URL redirects. A tutorial on the static site generator Guise Mummy
   is also included, illustrating how to generate a static site with
   clean resource names, host the site immediately using embedded
   Tomcat, and/or even deploy the site to AWS S3 using bucket object
   metadata with a single command.

Is that the sort of of level of talk they are looking for, and is it of 
appropriate general interest for the conference?


Garret

On 5/13/2019 1:28 PM, jean-frederic clere wrote:

Hi,

We have a small extension for The Tomcat track in ApacheConNA2019 CfP
until Tuesday 23h59 GMT! Go and submit now ;-)

determining resource MIME type via metadata sidecars

[Garret Wilson]

Let's say that I want to use embedded Tomcat to serve static content 
using `org.apache.catalina.servlets.DefaultServlet`. But rather than 
determining each resource's MIME type from the server's fixed list of 
MIME types, I want to determine each resource's MIME type dynamically 
based upon some resource metadata, not the resource filename extension.


Just as an example, for the resource `foobar`, I might have a 
`foobar.sidecar.properties` sidecar file that contains a `contentType` 
property of `text/html` for that resource, even though `foobar` has no 
extension at all.


(If you really want to know /why/: I'm experimenting having "clean" 
URLs, not by URL rewriting via Apache which I know how to do, but by 
uploading resources without extensions to AWS S3 buckets and setting the 
content type metadata for the resource there so they will be served as 
HTML files. But I still want to test locally with a server, which is 
where Tomcat comes in.)


So to pull this off, it looks like I would have to:

1. Extend `FileResource` so that it reads the metadata from wherever
   (e.g. a sidecar) and returns it via `getMimeType()`.
2. Extend `FileResourceSet` so that it creates `MyFileResource` instead
   of `FileResource`. (There doesn't seem to be a way to plug in a
   factory.)
3. Extend `StandardRoot` so that it creates `MyFileResourceSet` rather
   than `FileResourceSet`. (There doesn't seem to be a way to plug in a
   factory here, either.)
4. When I create my embedded context, call `myContext.setResources()`
   passing it `MyStandardRoot`.

Is that pretty much the gist of it?

This is probably going to be even more difficult than it sounds because 
some of the classes I have to extend don't nicely modularize their 
factory functionality. For example when I override 
`FileResourceSet.getResource()` to create a `MyFileResource`, I'll have 
to copy basically all the code in that method just to get at the single 
`return new FileResource(root, path, f, isReadOnly(), null)` line that I 
want to override. :(


Is this more trouble than it's worth? Is there an easier approach? Or is 
this another case where I should look into alternative embedded servers?


Garret

Re: no temp directory creation wanted when embedding Tomcat

[Garret Wilson]

On 4/24/2019 12:01 AM, Mark Thomas wrote:

If you don't want the baggage associated with a Servlet container it
might be worth looking at what other solutions are available.



Thanks for being forthright here, Mark. (Mark Wood also suggested the 
same thing separately.)


Indeed I have used Jetty in the past for embedding, and in fact that was 
my first thought for this project. But because I use Tomcat for my web 
applications, I thought maybe I had been ignoring Tomcat on this front. 
So I went the extra step of trying Tomcat for embedding, thinking maybe 
it would give me an opportunity to contribute back to this great project 
as well.


I guess I had assumed that Tomcat was "a great server that supports 
servlet technology", when in fact it is "a great servlet container that 
supports serving static files through a default servlet".


I'll think more about this. I have a question about dynamic MIME type 
determination, but I'll ask it separately.


Garret


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: no temp directory creation wanted when embedding Tomcat

[Garret Wilson]

On 4/23/2019 5:46 PM, Garret Wilson wrote:
3. Since the temporary working directory is on a per-context basis, 
and I've only set the "basedir" for the entire `Tomcat` instance, 
Tomcat must be determining a default temporary directory for the 
context. Surely I'm allowed to explicitly specify a temporary 
directory on a per-context basis, but I sure can't find it. The 
`StandardContext` seems to equate the "temporary directory" with the 
"work directory". Are they really the same? If so, I suppose I can 
just call `((StandardContext)context).setWorkDir(String)`?



I decided to just change the `Tomcat` base directory to some location. 
This will let Tomcat create its own per-context temp subdirectory under 
this, and it lowers the risk that I'll wind up with still other unwanted 
temporary directories in my project, which is under source control.


It turns out there's another trick here as well. You have to call 
`tomcat.setBaseDir()` /before/ you call the magic 
`tomcat.getConnector()` (the latter of which secretly creates the 
default connector). Otherwise, if the connector is created first, Tomcat 
decides on its own base directory using the current working directory 
and something in the form `tomcat:8080` (which is apparently why I got 
that particular directory subtree in the first place).


Garret

Re: no temp directory creation wanted when embedding Tomcat

[Garret Wilson]

On 4/22/2019 7:58 AM, Mark Thomas wrote:

On 21/04/2019 15:53, Garret Wilson wrote:

…
But now I realize Tomcat is creating this directory structur inside the
"base dir" I specified:

tomcat.8080/work/Tomcat/localhost/ROOT

I don't need this directory. I don't want this directory created. How do
I tell Tomcat not to create no danged directories?

The Servlet specification requires (see section 4.8.1 of the Servlet 4.0
spec) that Tomcat provides a private temporary directory for each
servlet context (web application).



I have have several reactions.

1. The first is, OK, so the servlet spec requires that the servlet 
container expose a temporary directory. Tomcat can't provide a temporary 
directory to the consumer unless it has a temporary directory, so it's 
reasonable that it creates one.


2. But so far nothing has been stored in this directory, so it's 
completely unused. I see also in the `Tomcat` class the comment, "lazy 
init for the temp dir - only when a JSP is compiled or get temp dir is 
called we need to create it. This will avoid the need for the baseDir". 
It sounds like you're lazily creating the JSP temporary directory, so 
why can't we do the same for the context temporary directory, which may 
never be used (and frankly probably won't be used in many applications)? 
But OK, that's a quibble; my other thoughts are more pressing.


3. Since the temporary working directory is on a per-context basis, and 
I've only set the "basedir" for the entire `Tomcat` instance, Tomcat 
must be determining a default temporary directory for the context. 
Surely I'm allowed to explicitly specify a temporary directory on a 
per-context basis, but I sure can't find it. The `StandardContext` seems 
to equate the "temporary directory" with the "work directory". Are they 
really the same? If so, I suppose I can just call 
`((StandardContext)context).setWorkDir(String)`?




The best you can do is specify an absolute path that already exists for
the Context's work directory. It doesn't need to be writeable. If you
are using it for multiple web applications you probably don't want it to
be writeable.



Ah, OK—I had read that response earlier, but didn't understand exactly 
what it meant until I investigated and wrote #2 above. So I guess I was 
right: I need to call`((StandardContext)context).setWorkDir(String)`.


4. But all this "temporary directory requirement" business is for 
servlet containers. I started out just wanted to embed Tomcat to serve 
static files; I don't (yet? ever?) need a servlet container for this 
particular application. So how can I use Tomcat to serve static files 
without bringing in the requirements of a servlet container?


Maybe the answer is that Tomcat serves files by using a 
static-file-serving servlet, which in turn requires me to use Tomcat as 
a servlet container even for serving static files, which in turn 
requires me to have a temporary directory I don't want or need.


I suppose that's just the way it is. This is not so much a complaint as 
a desire to confirm I understand correctly how things stands. (OK, it's 
a teeny-tiny complaint, too.)


Garret


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

no temp directory creation wanted when embedding Tomcat

[Garret Wilson]

As I mentioned in other emails, I am embedding Tomcat 9 (with OpenJDK 11 
on Windows 10) to serve static files from `/foo/bar`. Currently I'm not 
supporting Java webapps. I'm not supporting JSP. I just want to serve 
static files.


From your help in another thread, I called the magic 
"gets-but-really-creates-a-connection" method and got it working.


But now I realize Tomcat is creating this directory structur inside the 
"base dir" I specified:


tomcat.8080/work/Tomcat/localhost/ROOT

I don't need this directory. I don't want this directory created. How do 
I tell Tomcat not to create no danged directories?


Garret


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: unable to serve static files with embedded Tomcat

[Garret Wilson]

On 4/19/2019 8:04 PM, Rémy Maucherat wrote:

On Fri, Apr 19, 2019 at 11:14 PM Garret Wilson 
wrote:


On 4/19/2019 3:24 PM, Rémy Maucherat wrote:


tomcat.getService().addConnector(new Connector()); works very well, etc,
just look at what getConnector() does, it's very simple.


I did look at that first; that's why I mentioned the magic strings such 
as "HTTP/1.1". If I just use `new Connector()`, I may get something 
different than if I say `new Connector("HTTP/1.1")`. The latter 
potentially takes advantage of the APR connector if present; the former 
does not. (And how would I know that without looking at the source code?)


I'm happy to call `tomcat.getSerivce().addConnector(defaultConnector)`, 
but there needs to be some method for creating a connector with a 
default configuration. Otherwise I'm copying more and more large hunks 
of code from the Tomcat source code, which may break when things diverge 
in the future (not to mention cutting down on reusability).



…
You can also use a server.xml (and some other important config files like
web.xml) for your embedded now, you can look at Tomcat.main(...) (it is
used by the Tomcat container image example). If your configuration becomes
complex, it could be better than using Java to configure Tomcat.



I don't want server.xml or web.xml. This is /not/ complex; I merely want 
to serve static files in a directory. The irony is that I thought I was 
simplifying things my removing JSP etc. support; it turns out that this 
"simplification" complicated things greatly and required me to copy big 
hunks of Tomcat code. This could and should be improved, and I'm happy 
to help improve it.


Garret

Re: programmatically setting MIME mappings for static-only site

[Garret Wilson]

On 4/19/2019 5:12 PM, Christopher Schultz wrote:

On April 19, 2019 4:20:24 PM UTC, Garret Wilson
 wrote:


So I've probably answered my own question; this is an old TODO
that needs to be done, I suppose?

Yep.

+1

This is a perfect "beginner" enhancement.



Yay! I'd love to take that on.

Would you like to assign a GitHub ticket to me, or should I create one? 
I don't remember the details of the source control process, but you guys 
have described it before when talking about configuration. I'll review 
the other threads to remember what the GitHub status is, but if it's OK 
to assign me a GitHub ticket, that would sure be helpful (instead of my 
keeping track of emails).


Garret


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: unable to serve static files with embedded Tomcat

[Garret Wilson]

On 4/19/2019 3:24 PM, Rémy Maucherat wrote:

On Fri, Apr 19, 2019 at 7:46 PM Woonsan Ko  wrote:


I found this before from
https://stackoverflow.com/questions/48998387/code-works-with-embedded-apache-tomcat-8-but-not-with-9-whats-changed/49011424

 // Note: make sure that tomcat creates the default connector...
 tomcat.getConnector();

Worth trying...



Yes, that fixed it, Woonsan! Thank you so much!



That looks correct, and it's easy to spot with the logging (the connector
logs that it's starting if it's there). I removed the magic creation of the
connector because it was causing far more hacks (where the magic connector
had to be removed after its startup). It seemed more normal to have to
explicitly create a connector.

Rémy


I don't know the history of this, but I have a couple of doubts on the 
face of it.


First, the impression I get from reading _Apache Tomcat 7_ (Apress) is 
that I can create a server piecing together the components (Server, 
Service, Connector, Engine, Host, Context) if I want, but the whole 
point of the `Tomcat` class is to be a helper for doing all this 
automatically. So it seems a bit countertuitive that the `Tomcat` class 
is now making me do some of this wiring automatically. Why then would I 
use the `Tomcat` class? Why don't I just wire it all together from 
scratch? And why did we arbitrarily pick `Connector` for this "magic" 
creation? Why don't we have to call `Tomcat.getHost()` to make sure the 
host gets built, for instance?


Secondly, calling a getter method in order to invoke some secret magic 
creation sauce behind the scenes reeks of a kludge. (Nothing personal; 
I'm just commenting on the design.) This is not "normal". It's not at 
all obvious that there is this magic step required before the thing will 
work. For example, when you see the following code, without some blatant 
comment it's easy to think that this is a no-op call that should be removed:


    tomcat.setPort(8080);
    tomcat.getConnector();  //"It doesn't look like this does anything; 
maybe it we should remove it… "

    tomcat.setBaseDir("/foo/bar");

If I'm forced to create a connector, I would at least like the option to 
do something normal and expected, such as:


tomcat.getService().addConnector(tomcat.createDefaultConnector());

I _could_ create a default connector manually, but I'd have to be 
copying more of the source code (along with "magic" identifiers such as 
"HTTP/1.1"). So returning to the first doubt above: why am I using the 
`Tomcat` class if it doesn't do the default things for me automatically?


Garret


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

unable to serve static files with embedded Tomcat

[Garret Wilson]

Embedding Tomcat 9 (with OpenJDK 11 on Windows 10) I want to serve 
static files from `/foo/bar`. From scouring the web (primarily 
https://stackoverflow.com/a/15235711/421049 ), the documentation, and 
some books (primarily _Apache Tomcat 7_), I have this:


    Tomcat tomcat = new Tomcat();
    tomcat.setPort(8080);
    tomcat.setBaseDir("/foo");
    Context ctx = tomcat.addContext("", "/foo/bar");
    final Wrapper defaultServlet = ctx.createWrapper();
    defaultServlet.setName("default");
defaultServlet.setServletClass("org.apache.catalina.servlets.DefaultServlet");
    defaultServlet.addInitParameter("debug", "1");
    defaultServlet.addInitParameter("listings", "false");
    defaultServlet.setLoadOnStartup(1);
    ctx.addChild(defaultServlet);
    ctx.addServletMappingDecoded("/", "default");
    ctx.addWelcomeFile("index.html");
    tomcat.start();
    tomcat.getServer().await();

Everything looks like it starts up:

    Apr 19, 2019 2:18:09 PM org.apache.catalina.core.StandardService 
startInternal

    INFO: Starting service [Tomcat]
    Apr 19, 2019 2:18:09 PM org.apache.catalina.core.StandardEngine 
startInternal

    INFO: Starting Servlet engine: [Apache Tomcat/9.0.19]
    Apr 19, 2019 2:18:10 PM 
org.apache.catalina.util.SessionIdGeneratorBase createSecureRandom
    WARNING: Creation of SecureRandom instance for session ID 
generation using [SHA1PRNG] took [281] milliseconds.

    Apr 19, 2019 2:18:10 PM org.apache.catalina.core.ApplicationContext log
    INFO: default: DefaultServlet.init:  input buffer size=2048, output 
buffer size=2048


But connections to http://localhost:8080/ time out. (I'm pretty sure but 
not positive that I don't have anything blocking this in the firewall.)


Is there some simple thing I'm missing to connect things together?

Thanks,

Garret

P.S. The documentation on the web for the details of this is 
surprisingly sparse.



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

programmatically setting MIME mappings for static-only site

[Garret Wilson]

I'm wanting to embed Tomcat to only serve static files (for the moment). 
That is, no JSP, etc. I also want to have the welcome files completely 
customizable.


So instead of calling `tomcat.addWebapp()`, I go the completely 
programmable route and call `tomcat.addContext()`, 
`context.createWrapper()`, etc. This makes my bypass the 
`DefaultWebXmlListener`, which would have called 
`initWebappDefaults(Context ctx)` to set up the welcome files and such 
myself.


But `initWebappDefaults()` also sets up the default MIME mappings. And 
`Tomcat.DEFAULT_MIME_MAPPINGS` is private.


So the situation seems to be that Tomcat forces me to choose between 
creating a full-fledged JSP server, or setting up all the MIME types 
with some list of my own. Maybe it would be good for me to have my own 
list eventually, but for now this seems like an artificial choice forced 
upon me.


Part of the problem seems to be that the (ancient?) code has the MIME 
mappings as a string array!! Heaven knows we don't want to expose that. 
It should really be turned into a read-only map and then exposed so we 
can use it.


Then of course I see the comment:

> TODO: would a properties resource be better ? Or just parsing 
/etc/mime.types ?


To answer part of that question, we can't just parse `/etc/mime.types` 
because the embedded server might not even have an `/etc/mime.types` 
file. This should definitely be put into a properties resource, I would 
think.


So I've probably answered my own question; this is an old TODO that 
needs to be done, I suppose?


Garret


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: current best practices for Tomcat with SSL on port 443

[Garret Wilson]

On 2/7/2019 9:54 PM, Christopher Schultz wrote:

…
I would argue that adding Apache httpd into the mix (where is it not
already there) is more complicated than using Let's Encrypt with
Tomcat.



OK, I guess I didn't figure in the part about adding/configuring the 
connector. But still there are a few things I have doubts about, just 
looking over the document quickly:


 * There's still the issue about listening on lower port numbers. From
   the presentation, it looks like I would need to teach myself about
   iptables. I wonder if students (and I) would find mucking with
   iptable configurations easier than just installing apache using APT
   and editing some XML files. (I don't know; I haven't looked into it
   deeply.) And the presentation tantalizingly mentioned something
   called "jsvc" but didn't provide any further details. I'll have to
   research that. Then I'll search for "jsvc vs iptables", etc. So the
   presentation is a good thing to tell me what to look for.
 * What about forwarding from the non-secure site to the HTTPS site?
   Apache makes that pretty easy; actually it's a little arcane, but
   once you have the virtual host file one wants one can use it as a
   pattern. I'll note that the presentation didn't cover that. Or is
   that something iptables is responsible for, too?

Cheers,

Garret

Re: current best practices for Tomcat with SSL on port 443

[Garret Wilson]

On 2/7/2019 3:13 PM, Christopher Schultz wrote:

…
Have a look at this presentation:
https://people.apache.org/~schultz/ApacheCon%20NA%202018/Let's%20Encrypt
%20Apache%20Tomcat.pdf


The presentation gets two thumbs up, specifically:

 * Great corny grammar ambiguity joke on the title page.
 * The inferred conclusion of the presentation is: "If I already have
   enough complications in my life and I don't want more, I don't want
   to try to use Let's Encrypt directly with Tomcat (and especially not
   attempt to teach this to students), even if the complications can be
   overcome." :D But it's nice and useful to know what I'm avoiding by
   not going down that road.

Garret

current best practices for Tomcat with SSL on port 443

[Garret Wilson]

Hi, everyone. In the computer course I'm writing I'm using Tomcat for 
the server. (Students learn how to set up CentOS and everything from 
scratch. Currently the course has them using Tomcat running on port 
8080.) I'm going back to write the section on security. I want students 
to learn to set up their web server to use SSL/TLS on port 443, with 
HTTP port 80 redirecting to HTTPS port 443. This should be a very basic, 
fundamental configuration, no?


The last time I did this myself was about 10 or 15 years ago, when I 
compiled Apache myself and put it in front of Tomcat using whatever 
connectors (I'll have to go look at my configuration from back then), 
purchasing outrageously priced SSL certificates and installing them 
manually. How I'm sure things are greatly improved. Recently I've set up 
Apache (I didn't have to compile it) hosting static pages directly, and 
using Let's Encrypt (once I figured out what I should be doing) for SSL 
was a breeze. It's working nicely. So I assume I'd want to use Let's 
Encrypt in whatever solution I prescribe to the students.


So what is the best practice, straightforward, and simple setup for 
Tomcat with SSL on port 443 (preferably using Let's Encrypt) with HTTP 
port 80 forwarding to HTTPS port 443? Do I still need to stick Apache 
(or Nginx?) in front of it? (The last I checked, letting Tomcat use 
lower port numbers was a pain, and nobody seemed to know an easy, 
straightforward way to do it.)


Maybe this is a better question of Stack Overflow, but since the experts 
are here and I'm already on the list, I thought I'd ask. Thanks in 
advance! I'm really wanting to learn here.


Best,

Garret


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: latest situation with escaped path delimiters in URI

[Garret Wilson]

On 2/5/2019 1:15 PM, Mark Thomas wrote:

…
Migratation to git has been in planning for a while. We are pretty much
ready to pull the trigger. It is largely waiting for someone to have the
time to do it when there aren't other more urgent things to be dealt
with. I'd expect it to happen in the next few months.



That would really be super. If there are discussions that will help move 
this forward, I would like to contribute (to the discussions, at least) 
if it's helpful. About three years ago I migrated over a decade of my 
Subversion code to Git, and I did a lot of research on some of the 
gotchas. It involved splitting out branches and changing their roots. 
Here's the resulting script I used: 
https://bitbucket.org/globalmentor/util/src/master/bin/svn2git.sh (with 
related scripts in the same repo).


A not-unimportant factor is the whole LF vs CRLF nightmare. If you 
haven't been normalizing to LF in the Subversion repo from day one, 
you'll want to rewrite the entire Git history normalizing the text files 
(including source code) before using the repository. You'll want to add 
a good `.gitattributes` file so that this happens automatically going 
forward. (I understand you have this to some extent in Subversion 
already, but it's not as configurable as with `.gitattributes`.) Anyway, 
I hope you can go forward with this soon.



However, you can still use git and GitHub and provide PRs against
gitub.com/apache/tomcat  The ASF has integrations in place that make it
fairly easy for us to pull those in.



Oh, that's nice. Good news. OK, I'll start looking at the code, thanks.

Garret


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: latest situation with escaped path delimiters in URI

[Garret Wilson]

On 2/3/2019 9:34 PM, Mark Thomas wrote:



  * If this setting is still needed in some cases, is there any way to
    control it without resorting to a system property? (System
    properties are not very flexible, and Tomcat has many layers of more
    manipulable settings, as you all would know better than me.)

No. Moving system properties to more fine-grained configuration
locations is on the TODO list.



Is there really an actual to-do list, or were you speaking 
metaphorically? I'd be interested in looking at it.


I was crossing my fingers that https://github.com/apache/tomcat would 
actually be using the issue tracker, and that would be the to-do list, 
but I suppose I was expecting too much. Speaking of which, is the GitHub 
repository the canonical place for source code? Because 
https://tomcat.apache.org/svn.html claims that Tomcat still uses 
Suversion and that Git is just a read-only mirror, even though 
https://git.apache.org/ claims that some projects have switched to 
GitHub for their primary SCM.


I'm really interested in contributing, but I shudder at thinking about 
regressing a decade to start messing with Subversion (just 
`.gitattributes and how it more or less fixes the CRLF nightmare is a 
huge thing) and submitting patches rather than commenting on pull 
requests. (I have nothing against Subversion, mind you. I thought it was 
wonderful and I once used the property system extensively for per-file 
metadata in a custom CRM. But for a SCM… the world has moved on.) What's 
the Tomcat source code status?


Returning to the subject, do you have a list of system properties you'd 
be interested in making configurable, and would this be a welcome 
contribution? Where in the code code I look before committing myself?


Cheers,

Garret


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: latest situation with escaped path delimiters in URI

[Garret Wilson]

On 2/4/2019 7:31 PM, Christopher Schultz wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Garret,

On 2/3/19 16:20, Garret Wilson wrote:

If we want to look up the thing identified by
https://example.info/foobar, we would need to issue a request to
https://example.com/https%3A%2F%2Fexample.info%2Ffoobar/description

Why
are you %-encoding the slashes at all? They are perfectly legal as-is.



Hmmm… So let's say my RESTful API endpoint is 
https://example.com/{thingURI}/description as I mentioned. (Yes, I know 
that RESTful APIs don't have to be meaningful or structured as long as 
we use HATEOAS, but… a lot of us like them.) So you're saying that to 
request information for the resource https://example.info/foobar, I 
would send a GET request to:


https://example.com/https%3A//example.info/foobar/description

That raises all sorts of questions, such as

 * The double slash is OK? Really!??
 * Is there any RESTful API framework on the planet that would realize
   the URI path "/https%3A//example.info/foobar/description" matched
   "{thingURI}/description"? So if I'm using JAX-RS with a
   @Path("{thingURI}/description") with a string @PathParam("thingURI")
   thing, JAX-RS would set the "thing" parameter to
   "https://example.info/foobar;?? I highly doubt that.

Either I'm missing something and I'm going to learn something cool; or 
you missed some of the details of what I wrote. :) If I'm missing 
something, please explain because I'm ready to learn!


Garret

Re: latest situation with escaped path delimiters in URI

[Garret Wilson]

On 2/3/2019 3:34 PM, Mark Thomas wrote:

...
There is an open question what Tomcat should do with %2F sequences.


"What Tomcat should do" in what context? The servlet and JAX-RS specs 
may be clear about whether decoded or "raw" APIs should be returned from 
the various API methods. But I guess the issue here is /not/ whether 
JAX-RS should interpret a path segment as decoded or encoded. The issue 
is whether Tomcat has already fiddled with the URI itself to /change 
what constitutes the path segment/.


Unless an EE specification says to muck around with the URI like this, I 
don't see how the server has any business changing the content of the 
URI. If the escaped path delimiters are decoded early on, then the 
downstream APIs will get different path segments altogether: some will 
have characters missing, and there will moreover be additional path 
segments than intended. It would seem to be that "trying to be helpful 
without being asked" in this case (as in most cases) would probably 
raise security issues, too.


Further downstream, whether each API method returns encoded or decoded 
information would depend on what the API contracts say, for better or 
for worse.




It
currently decodes them. Arguably, it should leave them alone.


That sounds right to me.

I'll read the link you sent, thanks.

Garret

latest situation with escaped path delimiters in URI

[Garret Wilson]

Hi, all. I've stumbled on a situation I need some clarity on. As is 
typical, there's all sorts of information floating around, most of it 
more than a decade old, with no indication of what the current status is.


Our team is creating a RESTful API (using JAX-RS implemented by 
RESTEasy) to a general semantic framework in which each "thing" is 
identified by a URI. (The framework is URF , but that's 
a story for another day. It's analogous to RDF.) Basically we want to 
issue a GET to https://example.com/{thingURI}/description to get back 
info about the "thing".


If we want to look up the thing identified by 
https://example.info/foobar, we would need to issue a request to 
https://example.com/https%3A%2F%2Fexample.info%2Ffoobar/description . 
That should be completely legal and spec-compliant, and has been since 
web time began.


You no doubt already know the problem: Tomcat won't allow encoded 
slashes unless one sets system property 
org.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH to true. 
Apparently this there was a bug somewhere in Tomcat 6 
 
(back in 2007!) when used behind a proxy, as Mark explained on Stack 
Overflow . Tomcat 6 is 
really old, and Mark's Stack Overflow message seems to hint that it's 
not an issue anymore.


I'm not one to blindly change a setting unless I know what it's doing, 
and complain/advocate for change if it's no longer relevant. So I'm full 
of questions:


 * Is this even an issue anymore? Of not, is there a reason not to make
   org.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH default to true?
 * If this setting is still needed in some cases, is there any way to
   control it without resorting to a system property? (System
   properties are not very flexible, and Tomcat has many layers of more
   manipulable settings, as you all would know better than me.)
 * If we enable encoded slashes in Tomcat, do we need to do anything in
   Apache to get this to work if we put it in front of Tomcat? One
   really old Stack Overflow post
    indicated that there
   used to be a bug with AllowEncodedSlashes not being inherited by
   Apache virtual hosts. See also
   https://issues.sonatype.org/browse/NEXUS-10570 .
 * Do we need special configuration of mod_kj? (I haven't connected
   Apache to Tomcat in a while; I'm not sure the current best
   practices. I'll have to read up on that.) The connectors
   documentation
    is
   mentioning things like ForwardURIEscaped, which looks like it may be
   related.

I'm not even sure I asked all the right questions, but basically: I want 
to uses encoded slashes in my request URIs. What's the latest situation 
on that?


Thanks in advance,

Garret

Re: distinction between resource charset and format octet decoding

[Garret Wilson]

OK, Mark, I've made my initial edits to the 
https://wiki.apache.org/tomcat/FAQ/CharacterEncoding page. _Please check 
them over!_ This is my first edit to the wiki.


That page has a lot of legacy information, some of which had to do with 
internal Tomcat stuff, and some of which had to do with minute details 
of obsolete RFCs and evolution of browser behavior. I didn't want to 
spend the entire day (week?) on this, so I tried to surgically to only 
address the sections relating to POST of 
application/x-www-form-urlencoded and how percent-encoded octets are 
interpreted. I couldn't resist updating the specification links and 
changing just a little prose about URL percent encoding.


There is the risk now that other sections of the page are still outdated 
and conflict with my changes, but most importantly the FAQ should 
provide more complete information on how Tomcat web applications can be 
made to work with modern browsers.


Please let me know if I bungled anything or if I need to clarify something.

Thanks for letting me participate.

Garret

On 1/23/2019 12:26 AM, Mark Thomas wrote:

On 23/01/2019 05:07, Garret Wilson wrote:

On 1/15/2019 3:20 AM, Mark Thomas wrote:

…
Anything in PascalCase becomes a link to a wiki page of that name.
Usernames are created in this form so references to the user
automatically become links to that user's page in the wiki.


Ah, OK, that explains it. Very good to know. Maybe a little semantic
overloading, but as this is my first wiki account anywhere, I'm guessing
it's typical with whatever software you're using.

Anyway my account is created, with username `GarretWilson`. After I get
permissions I'll update the info on octet encoding for
application/x-www-form-urlencoded in relation to the servlet spec. It
may not be immediately, but I'll slowly but surely get to it.

Karma granted. Happy editing.

Cheers,

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: distinction between resource charset and format octet decoding

[Garret Wilson]

On 2/1/2019 9:38 AM, Christopher Schultz wrote:

Amazing. A close reading of RFC 3986 reveals that there is no
clear mandate for UTF-8 in existing URI schemes, even though
recommended for new schemes. Anyway, everyone seems to have settled
on UTF-8 (Tomcat included), so I'll try to indicate that.

Wait... are you saying that _it's the Wild West out there?_ ;)

Yes. The web is indeed held together with duct-tape and bailing wire.
It's amazing that it works as well as it does.



Hahaha. I'm /so/ happy someone agrees with me! Here's to improving 
things with a little JB Weld once in a while. (That's what my 
grandparents used on the farm when the bailing wire and duct tape 
couldn't handle it.)


Garret

Re: distinction between resource charset and format octet decoding

[Garret Wilson]

On 2/1/2019 7:23 AM, Garret Wilson wrote:

…
 * "There /is no default encoding for URIs/ specified anywhere, which
   is why there is a lot of confusion when it comes to decoding these
   values." Sheesh, this is is ancient. I'll correct it as per
   https://tools.ietf.org/html/rfc3986#section-2.5 .



Amazing. A close reading of RFC 3986 reveals that there is no clear 
mandate for UTF-8 in existing URI schemes, even though recommended for 
new schemes. Anyway, everyone seems to have settled on UTF-8 (Tomcat 
included), so I'll try to indicate that.


Garret


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: distinction between resource charset and format octet decoding

[Garret Wilson]

Good morning, I'm just getting to the editing. I'm going to list some 
thoughts I have as I go through this, so you can verify things:


 * The servlet spec links are way out of date. I'll update them.
 * "There /is no default encoding for URIs/ specified anywhere, which
   is why there is a lot of confusion when it comes to decoding these
   values." Sheesh, this is is ancient. I'll correct it as per
   https://tools.ietf.org/html/rfc3986#section-2.5 .
 * "Most of the web uses ISO-8859-1 as the default for query strings."
   Is this still true?! In light of the above, I would think it is not
   true, but I wanted to ask, as you know better about what you've seen
   "in the wild".

Garret

Eclipse freezes with Tomcat using request-character-encoding

[Garret Wilson]

We have a huge problem with Eclipse trying to simply handle Unicode 
characters in form submissions with Tomcat.


As we discussed in a separate thread, a modern browser will submit 
`application/x-www-form-urlencoded` forms with octects encoded from 
UTF-8 bytes (as it should as per HTML5), but the outdated Java Servlet 4 
specification requires the servlet container to interpret the octects as 
ISO-8859-1 bytes. The Servlet 4 mandated way to work around this is to 
add the following to the `WEB-INF/web.xml` file. (See 
https://stackoverflow.com/a/54098750/421049 if you need to refresh your 
memory.)


UTF-8

Basically if we don't do that, _the page will be broken, the submission 
will not be HTML5 compliant, and content will be corrupted_! That's 
pretty bad, in my opinion.


Unfortunately adding the above line to `WEB-INF/web.xml` seems to 
completely freeze Eclipse and generate a lot of errors. I won't include 
the whole stack trace here, but it stems from:


    org.eclipse.emf.ecore.xmi.FeatureNotFoundException: Feature 
'request-character-encoding' not found.


I've filed [Eclipse Bug 
543377](https://bugs.eclipse.org/bugs/show_bug.cgi?id=543377), but they 
are ignoring it. If anyone could contact someone at Eclipse and relate 
to them the importance of this, I'd appreciate it.


Garret

P.S. Sometimes it boggles my mind that fundamental pieces of the web 
infrastructure completely break with anything but ASCII—in 2019!



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: distinction between resource charset and format octet decoding

[Garret Wilson]

On 1/15/2019 3:20 AM, Mark Thomas wrote:

…
Anything in PascalCase becomes a link to a wiki page of that name.
Usernames are created in this form so references to the user
automatically become links to that user's page in the wiki.



Ah, OK, that explains it. Very good to know. Maybe a little semantic 
overloading, but as this is my first wiki account anywhere, I'm guessing 
it's typical with whatever software you're using.


Anyway my account is created, with username `GarretWilson`. After I get 
permissions I'll update the info on octet encoding for 
application/x-www-form-urlencoded in relation to the servlet spec. It 
may not be immediately, but I'll slowly but surely get to it.


Cheers,

Garret


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: distinction between resource charset and format octet decoding

[Garret Wilson]

On 1/9/2019 2:30 AM, Mark Thomas wrote:

…
Create yourself an account at https://wiki.apache.org/tomcat (click
login then create an account) and let the list know your ID. Then one of
the admins can add you to the allowed editors.



I was just ready to create an account, but I want to verify the details 
so I don't screw things up.


 * It asks for a "Name". Is this a username, I suppose? So we don't
   maintain our "name" separate from our "login username"?
 * It says to use "FirstnameLastName". Are you literally wanting us to
   use "JohnDoe", or can we use "johndoe"? Sorry for the questions; as
   one who works with protocols all the time, I automatically assume
   this stuff is important. But I prefer to use lowercase on my
   usernames; I'm a little confused about why this would want
   PascalCase for a login username. (I can't think of another system
   that I use that requires PascalCase usernames.)

My guess is that it's trying to maintain a "human name" and a "username" 
but combine them both into one field or something. I can't say this 
approach is typical…


Garret

Re: distinction between resource charset and format octet decoding

[Garret Wilson]

Hi, Mark, and thanks for some quick response. You provided some info I 
wasn't aware of. Some responses below:


On 1/8/2019 9:57 PM, Mark Thomas wrote:

On 08/01/2019 21:31, Garret Wilson wrote:



But as discussed above, this is completely wrong: the resource 
character encoding of a request sent in 
`application/x-www-form-urlencoded` should have absolutely no bearing 
on how the encoded octets within that resource are decoded.


That is not the correct interpretation of section 3.12 of the Servlet 
4.0 specification (note the section numbers do vary between spec 
versions). Tomcat implements the correct interpretation - i.e. the 
charset from the request content-type defines how encoded octets are 
decoded and, if none is specified, ISO-8859-1 is used as the default.



Ah, I hadn't seen that in the servlet spec. Yes, it seems as if Tomcat 
is correctly following the spec, but I would still say the servlet spec 
is wrong to make any linkage at all between resource encoding and %nn 
interpretation. In fact reading the prose it's not clear to me that the 
servlet spec is even strongly tying the %nn interpretation to the 
encoding. It just sees to say that, unless otherwise specified, the %nn 
interpretation should be ISO-8859-1. And actually that's a step up from 
the HTML 4.0.1 spec, which in 
https://www.w3.org/TR/html401/interact/forms.html#h-17.13.4.1 indicates 
that they should be interpreted as US-ASCII codes. :(


You indicate that this is all out of date, and I think we're in 
agreement there. We really, really need to get the next servlet 
specification to remove this part. In fact the servlet specification 
should defer to the official `application/x-www-form-urlencoded` 
specification, which at this point I think is the W3C HTML5 spec, which 
in turn defers to the WHATWG spec (which clearly says that UTF-8) should 
be used. What makes all of this more of a mess is that there seems to be 
no way to work around this from the client side, e.g. by putting 
something in the HTML to indicate UTF-8, as 
`application/x-www-form-urlencoded` doesn't support a `charset` parameter.


Anyway if there are any openings on the committee to update the servlet 
spec, let me know.




...
As of Servlet 4.0 there is a specification compliant configuration 
option to change this default to any encoding of your choice. 
Obviously, UTF-8 is one of the options. You can do this by adding the 
following to your web.xml:


UTF-8


Oh, that is really good to know, thanks!! Still I say that the request 
character encoding is orthogonal to the %nn encoding, but, still, it's 
good to have an implementation-agnostic way to do it.





Whether Tomcat should ship with this setting present in conf/web.xml 
by default is something that should probably be discussed for Tomcat 
10. Given the current state of the web, there is a reasonable case for 
doing so. I'll add that to the TOMCAT-NEXT discussion list.



Yes please! If I can help in any way, let me know.




The Tomcat Wiki also needs to be updated to take account of this new 
configuration option (and the related ). 
Since it is a wiki and this is clearly an issue you care about would 
you like to tackle that?



Yes, I'd love to. Let me know what permissions I need, etc.

I have an international flight boarding right now so I have to go, and I 
may not reply for the next few hours, but definitely sign me up.


Thanks,

Garret


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

distinction between resource charset and format octet decoding

[Garret Wilson]

I have question (using Tomcat 9.0.12 on Windows 10), and I'd like 
someone on the Tomcat development team to clarify a distinction for me 
regarding resource charsets and octet decoding in a particular format. I 
am not a newbie, and although the answer to my question may seem 
obvious, it goes to a critical issue that I believe to be a fundamental 
bug in Tomcat encoding processing.


Let's say that as an HTTP client I retrieve a resource `readme.txt` from 
Tomcat, and Tomcat clearly indicates via the HTTP response headers that 
the `Content-Type` is `text/plain; charset=ISO-8859-1`. That file 
contains, among things, a line that says:


    See https://example.com/example.jsp?fullName=Fl%C3%A1vio+Jos%C3%A9 
for more info.


I want parse the text file and present a live link to the user (email 
clients do this all the time), but I want to make the link "pretty" by 
decoding the URL. The question is: do I decode the octets using UTF-8, 
to show `…fullName=Flávio+José`, or do I use ISO-8859-1 to decode the 
octets, so that I show `…fullName=Flávio+José`? (Flávio José is a 
famous Brazilian forró singer and musician, by the way.)


The content type encoding of `readme.txt` is ISO-8859-1, so I must use 
ISO-8859-1 to decode the octets in `Fl%C3%A1vio+Jos%C3%A9`, yielding 
`…fullName=Flávio+José`, right??!


No, of course not. The decoding of the octet sequence is independent of 
the resource encoding, and represents a separate layer of encoding _on 
top_ of the resource encoding. It wouldn't matter whether the text file 
were encoded in UTF-8, ISO-8859-1, or US-ASCII—the URL would still be 
https://example.com/example.jsp?fullName=Fl%C3%A1vio+Jos%C3%A9, and its 
octets should still be decoded using UTF-8 as per RFC 3986.


I'll get right to the point; the above was a rhetorical question used as 
an analogy.


The Tomcat FAQ at 
https://wiki.apache.org/tomcat/FAQ/CharacterEncoding#Q9 indicates that 
the default encoding for an HTTP POST is ISO-8859-1. That is true. 
However Tomcat then goes further to then assume that it should decode 
_the octets of `application/x-www-form-urlencoded`_ using ISO-8859-1 as 
well! This is simply wrong; the octets should be interpreted as a 
sequence of UTF-8 octets; see 
https://url.spec.whatwg.org/#concept-urlencoded-serializer . This means 
if my browser sends a POST with content `fullName=Fl%C3%A1vio+Jos%C3%A9` 
using `application/x-www-form-urlencoded`, Tomcat will interpret this 
request parameter as `Flávio José` in my servlet/JSP, when it should 
interpret it as `Flávio José`. (Tomcat correctly decodes the octet when 
used as a query parameter rather than a POST parameter.)


Now it may be that the FAQ is simply out of date; it still seems to 
think that encoded URI octets should not be interpreted as UTF-8, 
completely ignoring RFC 3986. If so, it is long out of date; RFC 3986 
came out in 2005. (And indeed, Tomcat works with UTF-8 octets in URIs.) 
But out of date or not, the FAQ at 
https://wiki.apache.org/tomcat/FAQ/CharacterEncoding#Q8 then recommends 
that to force Tomcat to interpret the 
`application/x-www-form-urlencoded` octets as UTF-8, I must set the 
`org.apache.catalina.filters.SetCharacterEncodingFilter` filter (in some 
`web.xml` file) to `UTF-8`. (I can alternatively put `<% 
request.setCharacterEncoding("UTF-8"); %>` in my JSP.) And sure enough, 
it fixes the problem.


But as discussed above, this is completely wrong: the resource character 
encoding of a request sent in `application/x-www-form-urlencoded` should 
have absolutely no bearing on how the encoded octets within that 
resource are decoded. They must be decoded as UTF-8, irrespective of 
what "character encoding" Tomcat assumes the content to be. Tomcat has 
updated the way it decodes URIs to support UTF-8; it is time Tomcat does 
the same for `application/x-www-form-urlencoded` values. The current 
approach is broken in the context of the modern web, and the workaround 
is simply wrong.


I also raised this at https://stackoverflow.com/q/54094982/421049 .

I would have filed a Tomcat Bugzilla issue, but the bug report form 
indicated I should report the problem on this list first.


Thank you in advance for your attention to this matter.

Garret Wilson
GlobalMentor, Inc.


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Tomcat multiple init(), no init param

[Garret Wilson]

Tomcat 5.5.16 seems to call a servlet's init() method several times, and 
the specific init parameter is missing in all but the first.


I have a simple server.xml that includes:

Host name=www.example.com appBase=/var/web/apps
unpackWARs=false autoDeploy=false
xmlValidation=false xmlNamespaceAware=false

   Aliasexample.com/Alias

   Context path= docBase=www.example.com.war
   Parameter name=dataDirectory 
value=/var/web/data/www.example.com//

   /Context

/Host

In www.example.com.war I have a web.xml file that includes two instances 
of MyServlet, with servlet names of myServlet1 and myServlet2, mapped to 
/* and /example/*, respectively.


Tomcat does the following, which I find odd:

1. myServlet1.init() is called. The init parameter dataDirectory is 
correctly set to /var/web/data/www.example.com/.


2. catalina.out indicates Jun 25, 2006 2:07:26 PM 
org.apache.catalina.startup.HostConfig deployWAR.


3. myServlet.init is called(). The init parameter dataDirectory 
returns null.


4. myServlet2.init is called(). The init parameter dataDirectory 
returns null.


(Then steps 3 and 4 are repeated, presumably because steps 2 and 3 throw 
NullPointerExceptions, which results from the missing init parameter.)


This confuses me:

* Why is myServlet1.init() called twice?
* Why isn't myServlet2.init() called before the second call to 
myServlet1.init()?
* Why is the init parameter dataDirectory only available in the first 
call to myServlet1.init()?


Thanks for any insight,

Garret

-
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: Tomcat multiple init(), no init param

[Garret Wilson]

Everyone,

I found the answer to my second question, Why isn't myServlet2.init() 
called before the second call to myServlet1.init()? It turns out it was 
being called, but my debug trace statements had been redirected because 
my init() method had successfully redirected debug output.


So the sequence now looks like this:

1. myServlet1.init() is called. The init parameter dataDirectory is 
correctly set to /var/web/data/www.example.com/, and debug output is 
successfully redirected.


2. myServlet2.init() is called. The init parameter dataDirectory is 
correctly set to /var/web/data/www.example.com/, and debug output is 
still successfully redirected.


3. catalina.out indicates Jun 25, 2006 4:23:40 PM 
org.apache.catalina.startup.HostConfig deployWAR followed by INFO: 
Deploying web application archive www.example.com.war


4. myServlet1.init() is called. The init parameter dataDirectory 
returns null, the static variable is no longer set, and output is no 
longer redirected.


5. myServlet2.init() is called. The init parameter dataDirectory 
returns null, the static variable is no longer set, and output is no 
longer redirected..


Note that in step 1 I set a static variable in my servlet that is 
verified in step 2. But by step 4 the static variable is gone!


Note that if I change unpackWARs=true, these errors still occur, even 
if I restart Tomcat after the WARs have been unpacked! (The only 
difference is that, in step 4, I have access to the real path of the 
WEB-INF directory.)


It's as if two instances of Tomcat are being run in two separate JVMS: 
one using the server.xml configuration, and another attempting to deploy 
the .war files using some default configuration. The first set of 
initialized servlets continue to run and accept connections. What's 
wrong here?


Thanks in advance,

Garret

Garret Wilson wrote:
Tomcat 5.5.16 seems to call a servlet's init() method several times, 
and the specific init parameter is missing in all but the first.


I have a simple server.xml that includes:

Host name=www.example.com appBase=/var/web/apps
unpackWARs=false autoDeploy=false
xmlValidation=false xmlNamespaceAware=false

   Aliasexample.com/Alias

   Context path= docBase=www.example.com.war
   Parameter name=dataDirectory 
value=/var/web/data/www.example.com//

   /Context

/Host

In www.example.com.war I have a web.xml file that includes two 
instances of MyServlet, with servlet names of myServlet1 and 
myServlet2, mapped to /* and /example/*, respectively.


Tomcat does the following, which I find odd:

1. myServlet1.init() is called. The init parameter dataDirectory is 
correctly set to /var/web/data/www.example.com/.


2. catalina.out indicates Jun 25, 2006 2:07:26 PM 
org.apache.catalina.startup.HostConfig deployWAR.


3. myServlet.init is called(). The init parameter dataDirectory 
returns null.


4. myServlet2.init is called(). The init parameter dataDirectory 
returns null.


(Then steps 3 and 4 are repeated, presumably because steps 2 and 3 
throw NullPointerExceptions, which results from the missing init 
parameter.)


This confuses me:

* Why is myServlet1.init() called twice?
* Why isn't myServlet2.init() called before the second call to 
myServlet1.init()?
* Why is the init parameter dataDirectory only available in the 
first call to myServlet1.init()?


Thanks for any insight,

Garret

-
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



-
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: Tomcat multiple init(), no init param

[Garret Wilson]

Garret Wilson wrote:
It's as if two instances of Tomcat are being run in two separate JVMS: 
one using the server.xml configuration, and another attempting to 
deploy the .war files using some default configuration. The first set 
of initialized servlets continue to run and accept connections. What's 
wrong here?


I've found the problem. If you remember, my server.xml had this:

Host name=www.example.com appBase=/var/web/apps
unpackWARs=false autoDeploy=false
xmlValidation=false xmlNamespaceAware=false
   Aliasexample.com/Alias
   Context path= docBase=www.example.com.war
   Parameter name=dataDirectory 
value=/var/web/data/www.example.com//

   /Context
/Host

Tomcat would deploy my .war file as specified, using the given init 
parameter. But then, because my .war file was in the appBase directory 
(here /var/web/apps), Tomcat would going through and deploying the 
.war file a second time even though autoDeploy was set to false. One 
way to fix this it to change the appBase so that my .war file wasn't in 
the application base directory:


Host name=www.example.com appBase=/var/web
unpackWARs=false autoDeploy=false
xmlValidation=false xmlNamespaceAware=false
   Aliasexample.com/Alias
   Context path= docBase=apps/www.example.com.war
   Parameter name=dataDirectory 
value=/var/web/data/www.example.com//

   /Context
/Host

But the easier way is to add deployOnStartup=false to the host 
configuration, like this:


Host name=www.example.com appBase=/var/web/apps
unpackWARs=false autoDeploy=false deployOnStartup=false
xmlValidation=false xmlNamespaceAware=false
   Aliasexample.com/Alias
   Context path= docBase=www.example.com.war
   Parameter name=dataDirectory 
value=/var/web/data/www.example.com//

   /Context
/Host

Sure enough, there is a small note on this at 
http://tomcat.apache.org/tomcat-5.5-doc/config/host.html#Automatic%20Application%20Deployment 
: When using automatic deployment, the docBase defined by an XML 
Context file should be outside of the appBase directory. If this is not 
the case difficulties may be experienced deploying the web application 
or the application may be deployed twice.


Well, that's why I turned off autoDeploy---but it seems I need to turn 
off deployOnStartup as well.


Garret



Thanks in advance,

Garret

Garret Wilson wrote:
Tomcat 5.5.16 seems to call a servlet's init() method several times, 
and the specific init parameter is missing in all but the first.


I have a simple server.xml that includes:

Host name=www.example.com appBase=/var/web/apps
unpackWARs=false autoDeploy=false
xmlValidation=false xmlNamespaceAware=false

   Aliasexample.com/Alias

   Context path= docBase=www.example.com.war
   Parameter name=dataDirectory 
value=/var/web/data/www.example.com//

   /Context

/Host

In www.example.com.war I have a web.xml file that includes two 
instances of MyServlet, with servlet names of myServlet1 and 
myServlet2, mapped to /* and /example/*, respectively.


Tomcat does the following, which I find odd:

1. myServlet1.init() is called. The init parameter dataDirectory is 
correctly set to /var/web/data/www.example.com/.


2. catalina.out indicates Jun 25, 2006 2:07:26 PM 
org.apache.catalina.startup.HostConfig deployWAR.


3. myServlet.init is called(). The init parameter dataDirectory 
returns null.


4. myServlet2.init is called(). The init parameter dataDirectory 
returns null.


(Then steps 3 and 4 are repeated, presumably because steps 2 and 3 
throw NullPointerExceptions, which results from the missing init 
parameter.)


This confuses me:

* Why is myServlet1.init() called twice?
* Why isn't myServlet2.init() called before the second call to 
myServlet1.init()?
* Why is the init parameter dataDirectory only available in the 
first call to myServlet1.init()?


Thanks for any insight,

Garret

-
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



-
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



-
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

getting original URL

[Garret Wilson]

The darn Servlet API specification isn't making things easy for me.

How can I retrieve the path relative to the servlet of the HTTP request?

I can't use request.getPathInfo(), because this decodes the request URI. 
(No, I can't just re-encode it, because if the original URI included 
encoded slashes, this information would be lost.) I can't work backwards 
from request.getRequestURL(), because that method encodes the fragment 
identifier. (That is, if mypage.html#fragment were requested, this 
would return mypage.html%23fragment.)


Garret

P.S. To top things off, IE seems to send the #fragment in its request, 
but Mozilla doesn't. What am I doing wrong?


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]