date:20181031

[GUMP@vmgump-vm3]: Project tomcat-trunk-test-nio2 (in module tomcat-trunk) failed

2018-10-31 Thread Bill Barker

To whom it may engage...

This is an automated request, but not an unsolicited one. For 
more information please visit http://gump.apache.org/nagged.html, 
and/or contact the folk at gene...@gump.apache.org.

Project tomcat-trunk-test-nio2 has an issue affecting its community integration.
This issue affects 1 projects.
The current state of this project is 'Failed', with reason 'Build Failed'.
For reference only, the following projects are affected by this:
- tomcat-trunk-test-nio2 :  Tomcat 9.x, a web server implementing the Java 
Servlet 4.0,
...


Full details are available at:
http://vmgump-vm3.apache.org/tomcat-trunk/tomcat-trunk-test-nio2/index.html

That said, some information snippets are provided here.

The following annotations (debug/informational/warning/error messages) were 
provided:
 -DEBUG- Dependency on bnd exists, no need to add for property bndlib.jar.
 -INFO- Failed with reason build failed
 -INFO- Project Reports in: 
/srv/gump/public/workspace/tomcat-trunk/output/logs-NIO2
 -INFO- Project Reports in: 
/srv/gump/public/workspace/tomcat-trunk/output/test-tmp-NIO2/logs
 -WARNING- No directory 
[/srv/gump/public/workspace/tomcat-trunk/output/test-tmp-NIO2/logs]



The following work was performed:
http://vmgump-vm3.apache.org/tomcat-trunk/tomcat-trunk-test-nio2/gump_work/build_tomcat-trunk_tomcat-trunk-test-nio2.html
Work Name: build_tomcat-trunk_tomcat-trunk-test-nio2 (Type: Build)
Work ended in a state of : Failed
Elapsed: 24 mins 3 secs
Command Line: /usr/lib/jvm/java-8-oracle/bin/java -Djava.awt.headless=true 
-Dbuild.sysclasspath=only -Dsun.zip.disableMemoryMapping=true 
org.apache.tools.ant.Main -Dgump.merge=/srv/gump/public/gump/work/merge.xml 
-Djunit.jar=/srv/gump/public/workspace/junit/target/junit-4.13-SNAPSHOT.jar 
-Djava.net.preferIPv4Stack=/srv/gump/public/workspace/tomcat-trunk/true 
-Dobjenesis.jar=/srv/gump/public/workspace/objenesis/main/target/objenesis-3.1-SNAPSHOT.jar
 -Dtest.reports=output/logs-NIO2 -Dexecute.test.nio2=true 
-Dexamples.sources.skip=true 
-Dbase.path=/srv/gump/public/workspace/tomcat-trunk/tomcat-build-libs 
-Djdt.jar=/srv/gump/packages/eclipse/plugins/R-4.7.3a-201803300640/ecj-4.7.3a.jar
 -Dbndlib.jar=/srv/gump/packages/bnd/bndlib-4.0.0/biz.aQute.bndlib-4.0.0.jar 
-Dcommons-daemon.jar=/srv/gump/public/workspace/apache-commons/daemon/target/commons-daemon-1.1.1-SNAPSHOT.jar
 
-Dtest.openssl.path=/srv/gump/public/workspace/openssl-master/dest-20181101/bin/openssl
 -Dtest.temp=output/test-tmp-NIO2
  -Dtest.accesslog=true -Dexecute.test.nio=false 
-Dbnd.jar=/srv/gump/packages/bnd/bnd-4.0.0/biz.aQute.bnd-4.0.0.jar 
-Dexecute.test.apr=false -Dtest.excludePerformance=true -Dtest.relaxTiming=true 
-Deasymock.jar=/srv/gump/public/workspace/easymock/core/target/easymock-4.1-SNAPSHOT.jar
 -Dhamcrest.jar=/srv/gump/packages/hamcrest/hamcrest-core-1.3.jar 
-Dcglib.jar=/srv/gump/packages/cglib/cglib-nodep-2.2.jar test 
[Working Directory: /srv/gump/public/workspace/tomcat-trunk]
CLASSPATH: 
/usr/lib/jvm/java-8-oracle/lib/tools.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/webapps/examples/WEB-INF/classes:/srv/gump/public/workspace/tomcat-trunk/output/testclasses:/srv/gump/public/workspace/ant/dist/lib/ant.jar:/srv/gump/public/workspace/ant/dist/lib/ant-launcher.jar:/srv/gump/public/workspace/ant/dist/lib/ant-jmf.jar:/srv/gump/public/workspace/ant/dist/lib/ant-junit.jar:/srv/gump/public/workspace/ant/dist/lib/ant-junit4.jar:/srv/gump/public/workspace/ant/dist/lib/ant-swing.jar:/srv/gump/public/workspace/ant/dist/lib/ant-apache-resolver.jar:/srv/gump/public/workspace/ant/dist/lib/ant-apache-xalan2.jar:/srv/gump/public/workspace/xml-commons/java/build/resolver.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/bin/bootstrap.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/bin/tomcat-juli.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/annotations-api.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/servlet-api.ja
 
r:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/jsp-api.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/el-api.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/websocket-api.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/jaspic-api.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/catalina.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/catalina-ant.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/catalina-storeconfig.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/tomcat-coyote.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/jasper.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/jasper-el.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/catalina-tribes.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/catalina-ha.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/tomcat-api.jar:/srv/gump/public/workspace/tomcat-trunk/output/bu

[Bug 62873] New: Tomcat Monitor application will update multi-string values in Windows registry with blank lines

2018-10-31 Thread bugzilla

https://bz.apache.org/bugzilla/show_bug.cgi?id=62873

Bug ID: 62873
   Summary: Tomcat Monitor application will update multi-string
values in Windows registry with blank lines
   Product: Tomcat 8
   Version: 8.5.34
  Hardware: PC
Status: NEW
  Severity: normal
  Priority: P2
 Component: Meta
  Assignee: dev@tomcat.apache.org
  Reporter: brandon.wh...@tasktop.com
  Target Milestone: 

This is a Windows-platform specific issue.

The Tomcat Monitor application does not prevent a user from accidentally
inserting blank lines into the Java Options or Java 9 Options text areas. If a
blank line is introduced into either text area and the changes are applied (by
hitting the Apply or OK button), the changes are committed to the associating
multi-string value in the Windows registry.

If a user were to relaunch the Tomcat Monitor application and view the Java or
Java 9 Options again, they would see that the content has been clipped.
Individual options above the blank line would remain but any below would not be
present. If the Windows Registry editor were used to view the associating
multi-string values, they would see all of the content including the blank
line.

Changing the options again from within the Tomcat Monitor application will
succeed but when the options are committed to the registry multi-string value,
any options that existed below the blank line will be removed.

Note that the Windows registry editor will display an error dialog if a user
attempts to save a multi-string value with an empty line present. The message
is: "Data of type REG_MULTI_SZ cannot contain empty strings. Registry Editor
will remove all empty strings found."

Steps to Reproduce:

1) Download latest Tomcat 8 distribution for windows
2) Run service.bat install to install the default tomcat8 service.
3) Run tomcat8w.exe to start the tomcat monitor application. 
4) Switch to the Java tab and edit either the Java Options or Java 9 Options.
Insert a blank line somewhere in the middle of the text area. Click OK
5) Run tomcat8w.exe again. Look at the Java Options or Java 9 Options where the
blank line was inserted. Notice that all options below the blank line are not
visible.
6) Open up the registry editor and navigate to the multi-string value
associated with the Java (9) Options text area:
HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Apache Software Foundation\Procrun
2.0\Tomcat8\Parameters\Java
7) Notice that all content is present (including the blank line) in the
registry value.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

[GitHub] tomcat pull request #130: Add two spaces (nbsp) in front of the Undeploy but...

2018-10-31 Thread ljader

GitHub user ljader opened a pull request:

https://github.com/apache/tomcat/pull/130

Add two spaces (nbsp) in front of the Undeploy button

Since there is no JavaScript in the Manager HTML,
I decided to visually separate the Undeploy button from the rest of the 
buttons.

This is to "prevent" **missclicks** on the Reload button,
which ends with the Undeploy being clicked.

(there is no Bugzilla ticket for this - I've created this to get your 
opinion on this.)

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ljader/tomcat trunk

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/tomcat/pull/130.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #130


commit a775d89f16563de3959189baa6b74acfd6696419
Author: Lukasz Jader 
Date:   2018-11-01T00:12:12Z

Add two spaces (nbsp) in front of the Undeploy button

This is to "prevent" missclicks on the Reload button,
which ends with the Undeploy being clicked.




---

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Mime types

2018-10-31 Thread Rémy Maucherat

On Wed, Oct 31, 2018 at 10:30 PM Igal Sapir  wrote:

> >> Unless DefaultServlet behaviour is fixed, enabling probeContentType is
> >> likely to break my configurations.
>
> How about allowing to remove a Mime type by mapping it to an empty
> string?  That way you would only need to override that one in your
> Application's web.xml.
>
> Would that work?
>

That won't work, but I'm adding an init-param for the feature to disable it
at the same time.


>
> >> 4. I see a similarity to mod_mime_magic module of HTTPD.
> >>
> >> http://httpd.apache.org/docs/current/mod/mod_mime.html
> >> http://httpd.apache.org/docs/current/mod/mod_mime_magic.html
> >>
> >> (For some reason I though that mod_mime_magic uses the magic file from
> >> Unix OS.
> >> Actually it uses its own magic file from configuration of HTTPD,
> >> configured by directive "MimeMagicFile".
> >> So it is actually portable.)
> >>
> >> MimeMagicFile directive is disabled by default.
> >>
> >>
> http://svn.apache.org/viewvc/httpd/httpd/trunk/docs/conf/httpd.conf.in?view=markup#l330
> >>
> >> Performance =?
> >> The documentation of mod_mime_magic says that performance is a concern
> >> for this module.
>
> The lookup will only happen for unknown types, and as Remy said they
> will be cached so I don't think that performance is an issue here.
>
> >
> > No idea really, probeContentType checks a number of sources [on Fedora it
> > is: new GnomeFileTypeDetector(), new
> > MimeTypesFileTypeDetector(userMimeTypes), new
> > MimeTypesFileTypeDetector(etcMimeTypes), new MagicFileTypeDetector()],
> and
> > it is platform dependent as you say. The result is then cached into the
> > resource in Tomcat, so it should be "ok" performance wise. The question
> is:
> > besides the default servlet not respecting a possibly set content-type (I
> > have not looked at it, and IMO it is a separate issue, no problem if
> you'd
> > like to fix it), is the Files.probeContentType result good enough on
> > Windows ?
>
> I have both Windows and Fedora so if you want me to test something
> specific please let me know.
>

Looked at the code, Windows uses the registry only (so something that may
be incomplete and not standardized). As a result, it would be unreasonable
to remove the current default mappings.
Example: https://bugs.java.com/bugdatabase/view_bug.do?bug_id=8080369

Rémy

[Bug 62871] New: JMX: ThreadPool.socketProperties is not Serializable

2018-10-31 Thread bugzilla

https://bz.apache.org/bugzilla/show_bug.cgi?id=62871

Bug ID: 62871
   Summary: JMX: ThreadPool.socketProperties is not Serializable
   Product: Tomcat 8
   Version: 8.5.x-trunk
  Hardware: All
OS: All
Status: NEW
  Severity: normal
  Priority: P2
 Component: Util
  Assignee: dev@tomcat.apache.org
  Reporter: m...@suse.de
  Target Milestone: 

Created attachment 36224
  --> https://bz.apache.org/bugzilla/attachment.cgi?id=36224=edit
Trivial patch that addresses this bug

Some JMX clients such as Prometheus'[1] jmx_exporter[2] will bulk-load all
attributes of an MBean for performance reasons. This means that one
non-Serializable attribute is sufficient to make all information from the MBean
inaccessible.


I could readily reproduce a problem with the ThreadPool MBean, specifically
with the socketProperties property which is not Serializable.

Reproducer:

# enable JMX's full logging
cat >logging.properties <<<"
handlers=java.util.logging.ConsoleHandler
java.util.logging.ConsoleHandler.level=ALL
io.prometheus.jmx.level=ALL
io.prometheus.jmx.shaded.io.prometheus.jmx.level=ALL
"

# configure reading from a local Tomcat instance with JMX configured on port

cat >config.yml <<<"
hostPort: localhost:
whitelistObjectNames:
  - Catalina:type=ThreadPool,name=*
rules:
  - pattern: ".*"
"

# launch jmx_exporter
/usr/bin/java -Djava.util.logging.config.file=logging.properties -jar
jmx_prometheus_httpserver.jar 5556 jmx_exporter.yml


Visiting with a browser http://localhost:5556/metrics will trigger jmx_exporter
and the following error is produced:

FINE: scrape:
'Catalina:type=ThreadPool,name="http-apr-127.0.0.1-8080"_[currentThreadsBusy,
paused, sSLCertificateKeyFile, sSLProtocol, keyAlias, truststoreType,
sSLPassword, sendfileCount, sSLCACertificatePath, useCometTimeout,
socketProperties, p
ollTime, sSLHonorCipherOrder, currentThreadCount, trustManagerClassName,
sSLCARevocationFile, usePolling, keystorePass, trustMaxCertLength, maxThreads,
truststoreAlgorithm, tcpNoDelay, algorithm, keepAliveTimeout,
maxKeepAliveRequests, localPort, t
ruststoreProvider, acceptorThreadCount, soTimeout, daemon, minSpareThreads,
sSLDisableSessionTickets, sessionCacheSize, acceptorThreadPriority, backlog,
port, maxHeaderCount, sSLCARevocationPath, keystoreType, name, sslProtocol,
soLinger, sessionTi
meout, useComet, sSLVerifyDepth, clientAuth, modelerType, sSLCertificateFile,
connectionCount, sendfileThreadCount, threadPriority, running,
executorTerminationTimeoutMillis, allowUnsafeLegacyRenegotiation,
sslEnabledProtocolsArray, jniSslContext,
ciphers, truststorePass, sSLEnabled, sendfileSize, maxThreadsWithExecutor,
crlFile, sSLVerifyClient, maxConnections, keystoreProvider, keepAliveCount,
sSLCipherSuite, deferAccept, sSLDisableCompression, useSendfile,
sSLCertificateChainFile, sSLCACe
rtificateFile, ciphersUsed, keyPass, bindOnInit, sSLInsecureRenegotiation,
truststoreFile, keystoreFile, useServerCipherSuitesOrder]': Fail:
java.rmi.UnmarshalException: error unmarshalling return; nested exception is:
java.io.WriteAbortedException: writing aborted;
java.io.NotSerializableException: org.apache.tomcat.util.net.SocketProperties


I have successfully tested a trivial patch that simply makes socketProperties
protected instead of public - this does not seem to have negative consequences
at least in the environment I tested it in (Tomcat 8.0.53 as shipped in SUSE
Linux Enterprise Server 12 SP3). Patch is attached to this bug.


I do not really know Tomcat (and not even JMX, to be honest) to know if this
patch is really acceptable or if it's really oversimplified, as this is my
first contribution!


[1] https://prometheus.io/
[2] https://github.com/prometheus/jmx_exporter
[3]
https://github.com/prometheus/jmx_exporter/issues/265#issuecomment-383614412

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Mime types

2018-10-31 Thread Igal Sapir

On 10/31/2018 12:50 PM, Rémy Maucherat wrote:

On Wed, Oct 31, 2018 at 8:16 PM Konstantin Kolinko
wrote:

ср, 31 окт. 2018 г. в 19:38, Rémy Maucherat :

Hi,

There are two main contraptions in Tomcat that do (badly ...) extension tomime
type mapping: the shared web.xml and some hardcoded stuff in
startup.Tomcat.

While we should obviously have support for user configured mime types in
web.xml, as it's the spec, there should be a possibility to use
Files.probeContentType as the fallback when a mime type isn't found (and
maybe also have an option to disable it ? - although I don't quite see why it
would bother anyone). After looking at its implementation, it looks into all
mime type locations we might want (the OS, a mime.types file, etc). Theonly
problem is that it uses a Path (that would be an issue since it's
super tied to a real filesystem), but thankfully it mostly uses toSting and
thus can be worked around using a new fake Path implementation.

The code calling Files.probeContentType could be inserted here in
DefaultServlet:
// Find content type.
String contentType = resource.getMimeType();
if (contentType == null) {
contentType =
getServletContext().getMimeType(resource.getName());
--->
resource.setMimeType(contentType);
}

And then all the badly maintained content from web.xml and the Tomcat class can
be deleted.

Comments ?

1. "badly maintained content from web.xml"

Do not call them "bad".

Ok, but they didn't look too good. My mime.types has a lot more types for
starters, and it's so big I don't feel like adding all that.

AFAIK, Those are synchronized with httpd. IIRC there was python script
to check the sync. Technically, it should be possible to sync with
IANA registry.

(I do not remember the details - those should be easy to find in the
archives of this mailing list. I just remember that the last time that
the sync was checked, there was some good job done to automate and
perform the check.)

Who maintains the mappings used Files.probeContentType and why do you
think that those are maintained any better?

I remember we had a number of BZs asking to add or fix mime mappings. Annoying.

If an OS is an LTS one, are mime-mapping configurations in the OS
updated as the time goes?

Well, ultimately the guy can still add its new mapping in web.xml. Just the
basic usual ones don't need to be there.

It should be possible to write a JUnit test to keep the mappings in
startup.Tomcat.DEFAULT_MIME_MAPPINGS in sync with the default web.xml
file.

2. web.xml is portable between operating systems.

I would expect surprises from Files.probeContentType()

Looking at javadocs, FileTypeDetector is pluggable,
and the default behaviour is os-dependent

https://docs.oracle.com/javase/8/docs/api/java/nio/file/Files.html#probeContentType-java.nio.file.Path-

https://docs.oracle.com/javase/8/docs/api/java/nio/file/spi/FileTypeDetector.html

3, My own story: I had to remove the default mime-type mapping for
"gz" from conf/web.xml in my configurations:

For filenames like "filename.foo.gz" it is "foo" part that determines
the mime-type for me. This cannot be configured in web.xml. I use a
filter (urlrewrite) to set content-type for requests to those files.

Generally, configuring a Filter should have been enough. But there is
a bug in the DefaultServlet that it does not respect the content-type
that has already been set on the response and blindly overwrites it.
Unless I remove the default mapping for "gz", the content-type value
set by a filter is overwritten.

Unless DefaultServlet behaviour is fixed, enabling probeContentType is
likely to break my configurations.

How about allowing to remove a Mime type by mapping it to an empty
string? That way you would only need to override that one in your
Application's web.xml.

Would that work?

4. I see a similarity to mod_mime_magic module of HTTPD.

http://httpd.apache.org/docs/current/mod/mod_mime.html
http://httpd.apache.org/docs/current/mod/mod_mime_magic.html

(For some reason I though that mod_mime_magic uses the magic file from
Unix OS.
Actually it uses its own magic file from configuration of HTTPD,
configured by directive "MimeMagicFile".
So it is actually portable.)

MimeMagicFile directive is disabled by default.

http://svn.apache.org/viewvc/httpd/httpd/trunk/docs/conf/httpd.conf.in?view=markup#l330

Performance =?
The documentation of mod_mime_magic says that performance is a concern
for this module.

The lookup will only happen for unknown types, and as Remy said they
will be cached so I don't think that performance is an issue here.

No idea really, probeContentType checks a number of sources [on Fedora it
is: new GnomeFileTypeDetector(), new
MimeTypesFileTypeDetector(userMimeTypes), new
MimeTypesFileTypeDetector(etcMimeTypes), new MagicFileTypeDetector()], and
it is platform dependent as you say. The result is then cached into the
resource in Tomcat, so it should be

Re: Mime types

2018-10-31 Thread Rémy Maucherat

On Wed, Oct 31, 2018 at 8:16 PM Konstantin Kolinko 
wrote:

> ср, 31 окт. 2018 г. в 19:38, Rémy Maucherat :
> >
> > Hi,
> >
> > There are two main contraptions in Tomcat that do (badly ...) extension
> to
> > mime type mapping: the shared web.xml and some hardcoded stuff in
> > startup.Tomcat.
> >
> > While we should obviously have support for user configured mime types in
> > web.xml, as it's the spec, there should be a possibility to use
> > Files.probeContentType as the fallback when a mime type isn't found (and
> > maybe also have an option to disable it ? - although I don't quite see
> why
> > it would bother anyone). After looking at its implementation, it looks
> into
> > all mime type locations we might want (the OS, a mime.types file, etc).
> The
> > only problem is that it uses a Path (that would be an issue since it's
> > super tied to a real filesystem), but thankfully it mostly uses toSting
> and
> > thus can be worked around using a new fake Path implementation.
> >
> > The code calling Files.probeContentType could be inserted here in
> > DefaultServlet:
> > // Find content type.
> > String contentType = resource.getMimeType();
> > if (contentType == null) {
> > contentType =
> > getServletContext().getMimeType(resource.getName());
> > --->
> > resource.setMimeType(contentType);
> > }
> >
> > And then all the badly maintained content from web.xml and the Tomcat
> class
> > can be deleted.
> >
> > Comments ?
>
> 1. "badly maintained content from web.xml"
>
> Do not call them "bad".
>

Ok, but they didn't look too good. My mime.types has a lot more types for
starters, and it's so big I don't feel like adding all that.

>
> AFAIK, Those are synchronized with httpd. IIRC there was python script
> to check the sync. Technically, it should be possible to sync with
> IANA registry.
>
> (I do not remember the details - those should be easy to find in the
> archives of this mailing list. I just remember that the last time that
> the sync was checked, there was some good job done to automate and
> perform the check.)
>
> Who maintains the mappings used Files.probeContentType and why do you
> think that those are maintained any better?
>

I remember we had a number of BZs asking to add or fix mime mappings.
Annoying.

>
> If an OS is an LTS one, are mime-mapping configurations in the OS
> updated as the time goes?
>

Well, ultimately the guy can still add its new mapping in web.xml. Just the
basic usual ones don't need to be there.

>
> It should be possible to write a JUnit test to keep the mappings in
> startup.Tomcat.DEFAULT_MIME_MAPPINGS in sync with the default web.xml
> file.
>
> 2. web.xml is portable between operating systems.
>
> I would expect surprises from Files.probeContentType()
>
> Looking at javadocs, FileTypeDetector is pluggable,
> and the default behaviour is os-dependent
>
>
> https://docs.oracle.com/javase/8/docs/api/java/nio/file/Files.html#probeContentType-java.nio.file.Path-
>
> https://docs.oracle.com/javase/8/docs/api/java/nio/file/spi/FileTypeDetector.html
>
> 3, My own story: I had to remove the default mime-type mapping for
> "gz" from conf/web.xml in my configurations:
>
> For filenames like "filename.foo.gz" it is "foo" part that determines
> the mime-type for me. This cannot be configured in web.xml. I use a
> filter (urlrewrite) to set content-type for requests to those files.
>
> Generally, configuring a Filter should have been enough. But there is
> a bug in the DefaultServlet that it does not respect the content-type
> that has already been set on the response and blindly overwrites it.
> Unless I remove the default mapping for "gz", the content-type value
> set by a filter is overwritten.
>
> Unless DefaultServlet behaviour is fixed, enabling probeContentType is
> likely to break my configurations.
>
> 4. I see a similarity to mod_mime_magic module of HTTPD.
>
> http://httpd.apache.org/docs/current/mod/mod_mime.html
> http://httpd.apache.org/docs/current/mod/mod_mime_magic.html
>
> (For some reason I though that mod_mime_magic uses the magic file from
> Unix OS.
> Actually it uses its own magic file from configuration of HTTPD,
> configured by directive "MimeMagicFile".
> So it is actually portable.)
>
> MimeMagicFile directive is disabled by default.
>
> http://svn.apache.org/viewvc/httpd/httpd/trunk/docs/conf/httpd.conf.in?view=markup#l330
>
> Performance =?
> The documentation of mod_mime_magic says that performance is a concern
> for this module.
>

No idea really, probeContentType checks a number of sources [on Fedora it
is: new GnomeFileTypeDetector(), new
MimeTypesFileTypeDetector(userMimeTypes), new
MimeTypesFileTypeDetector(etcMimeTypes), new MagicFileTypeDetector()], and
it is platform dependent as you say. The result is then cached into the
resource in Tomcat, so it should be "ok" performance wise. The question is:
besides the default servlet not respecting a possibly set content-type (I
have not

Re: Mime types

2018-10-31 Thread Konstantin Kolinko

ср, 31 окт. 2018 г. в 19:38, Rémy Maucherat :
>
> Hi,
>
> There are two main contraptions in Tomcat that do (badly ...) extension to
> mime type mapping: the shared web.xml and some hardcoded stuff in
> startup.Tomcat.
>
> While we should obviously have support for user configured mime types in
> web.xml, as it's the spec, there should be a possibility to use
> Files.probeContentType as the fallback when a mime type isn't found (and
> maybe also have an option to disable it ? - although I don't quite see why
> it would bother anyone). After looking at its implementation, it looks into
> all mime type locations we might want (the OS, a mime.types file, etc). The
> only problem is that it uses a Path (that would be an issue since it's
> super tied to a real filesystem), but thankfully it mostly uses toSting and
> thus can be worked around using a new fake Path implementation.
>
> The code calling Files.probeContentType could be inserted here in
> DefaultServlet:
> // Find content type.
> String contentType = resource.getMimeType();
> if (contentType == null) {
> contentType =
> getServletContext().getMimeType(resource.getName());
> --->
> resource.setMimeType(contentType);
> }
>
> And then all the badly maintained content from web.xml and the Tomcat class
> can be deleted.
>
> Comments ?

1. "badly maintained content from web.xml"

Do not call them "bad".

AFAIK, Those are synchronized with httpd. IIRC there was python script
to check the sync. Technically, it should be possible to sync with
IANA registry.

(I do not remember the details - those should be easy to find in the
archives of this mailing list. I just remember that the last time that
the sync was checked, there was some good job done to automate and
perform the check.)

Who maintains the mappings used Files.probeContentType and why do you
think that those are maintained any better?

If an OS is an LTS one, are mime-mapping configurations in the OS
updated as the time goes?

It should be possible to write a JUnit test to keep the mappings in
startup.Tomcat.DEFAULT_MIME_MAPPINGS in sync with the default web.xml
file.

2. web.xml is portable between operating systems.

I would expect surprises from Files.probeContentType()

Looking at javadocs, FileTypeDetector is pluggable,
and the default behaviour is os-dependent

https://docs.oracle.com/javase/8/docs/api/java/nio/file/Files.html#probeContentType-java.nio.file.Path-
https://docs.oracle.com/javase/8/docs/api/java/nio/file/spi/FileTypeDetector.html

3, My own story: I had to remove the default mime-type mapping for
"gz" from conf/web.xml in my configurations:

For filenames like "filename.foo.gz" it is "foo" part that determines
the mime-type for me. This cannot be configured in web.xml. I use a
filter (urlrewrite) to set content-type for requests to those files.

Generally, configuring a Filter should have been enough. But there is
a bug in the DefaultServlet that it does not respect the content-type
that has already been set on the response and blindly overwrites it.
Unless I remove the default mapping for "gz", the content-type value
set by a filter is overwritten.

Unless DefaultServlet behaviour is fixed, enabling probeContentType is
likely to break my configurations.

4. I see a similarity to mod_mime_magic module of HTTPD.

http://httpd.apache.org/docs/current/mod/mod_mime.html
http://httpd.apache.org/docs/current/mod/mod_mime_magic.html

(For some reason I though that mod_mime_magic uses the magic file from Unix OS.
Actually it uses its own magic file from configuration of HTTPD,
configured by directive "MimeMagicFile".
So it is actually portable.)

MimeMagicFile directive is disabled by default.
http://svn.apache.org/viewvc/httpd/httpd/trunk/docs/conf/httpd.conf.in?view=markup#l330

Performance =?
The documentation of mod_mime_magic says that performance is a concern
for this module.

Best regards,
Konstantin Kolinko

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

37 matches

Mail list logo