Re: Embedded tomcat does not find web-fragment in jars outside web-inf\lib continued...

[Mark Thomas]

On 04/10/17 08:53, Brian Toal wrote:
> The chain [1] left of with:
> "The relevant language is in section 8.2.1
> 
> 
> If a framework wants its META-INF/web-fragment.xml honored in such a way
> that it augments a web application's web.xml, the framework must be bundled
> within the web application's WEB-INF/lib directory
> 
> 
> Therefore, Tomcat 8.0 looks to be doing the right thing."
> 
> Which I think is fine for the case where the container is augmenting a
> existing web.xml.
> 
> However the spec also says "If a framework is packaged as a jar file and
> has metadata information in the form of deployment descriptor then the
> web-fragment.xml descriptor must be in the META-INF/ directory of the jar
> file."
> 
> I'm not interested in augmenting a existing web.xml, nor my embedded app is
> using WEB-INF/lib, which seems not to meet the eligibility critieria for
> adding the jar to WEB-INF/lib , tomcat should honor
> META-INF/web-fragment.xml in jars in the absence of web.xml, to determine
> if the Servlet 3.0 annotations are going to be scanned.  This is the
> problem I'm having after fixing the classloading issues in [2].
> 
> Thoughts?

If you use addContext() the expectation is that you also take on
responsibility for performing all of the configuration.

If you want all the standard Servlet spec behaviour out of the box, you
need to use addWebapp().

If you take the addContext() route then you can make use of the same
code Tomcat uses to perform the configuration. Suggestions on how to
make that more extensible / flexible / etc. welcome.

Mark


> [1] - https://marc.info/?l=tomcat-user=149443917508077=2
> [2] - https://marc.info/?l=tomcat-user=150701409822162=2
> 


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Embedded tomcat does not find web-fragment in jars outside web-inf\lib continued...

[Brian Toal]

Jetty also makes it very easy to scan jar for @WebServlet, @WebFilter,
@WebListener via AnnotationConfiguration.

http://www.eclipse.org/jetty/documentation/9.4.x/configuring-webapps.html

On Wed, Oct 4, 2017 at 12:53 AM, Brian Toal  wrote:

> The chain [1] left of with:
> "The relevant language is in section 8.2.1
>
> 
> If a framework wants its META-INF/web-fragment.xml honored in such a way
> that it augments a web application's web.xml, the framework must be bundled
> within the web application's WEB-INF/lib directory
> 
>
> Therefore, Tomcat 8.0 looks to be doing the right thing."
>
> Which I think is fine for the case where the container is augmenting a
> existing web.xml.
>
> However the spec also says "If a framework is packaged as a jar file and
> has metadata information in the form of deployment descriptor then the
> web-fragment.xml descriptor must be in the META-INF/ directory of the jar
> file."
>
> I'm not interested in augmenting a existing web.xml, nor my embedded app
> is using WEB-INF/lib, which seems not to meet the eligibility critieria for
> adding the jar to WEB-INF/lib , tomcat should honor
> META-INF/web-fragment.xml in jars in the absence of web.xml, to determine
> if the Servlet 3.0 annotations are going to be scanned.  This is the
> problem I'm having after fixing the classloading issues in [2].
>
> Thoughts?
>
>
> [1] - https://marc.info/?l=tomcat-user=149443917508077=2
> [2] - https://marc.info/?l=tomcat-user=150701409822162=2
>
>
>

Embedded tomcat does not find web-fragment in jars outside web-inf\lib continued...

[Brian Toal]

The chain [1] left of with:
"The relevant language is in section 8.2.1


If a framework wants its META-INF/web-fragment.xml honored in such a way
that it augments a web application's web.xml, the framework must be bundled
within the web application's WEB-INF/lib directory


Therefore, Tomcat 8.0 looks to be doing the right thing."

Which I think is fine for the case where the container is augmenting a
existing web.xml.

However the spec also says "If a framework is packaged as a jar file and
has metadata information in the form of deployment descriptor then the
web-fragment.xml descriptor must be in the META-INF/ directory of the jar
file."

I'm not interested in augmenting a existing web.xml, nor my embedded app is
using WEB-INF/lib, which seems not to meet the eligibility critieria for
adding the jar to WEB-INF/lib , tomcat should honor
META-INF/web-fragment.xml in jars in the absence of web.xml, to determine
if the Servlet 3.0 annotations are going to be scanned.  This is the
problem I'm having after fixing the classloading issues in [2].

Thoughts?


[1] - https://marc.info/?l=tomcat-user=149443917508077=2
[2] - https://marc.info/?l=tomcat-user=150701409822162=2

Re: Embedded tomcat does not find web-fragment in jars outside web-inf\lib

[Michael Heinen]

Am 10.05.2017 um 19:59 schrieb Mark Thomas:

On 10/05/17 12:40, Michael Heinen wrote:

Am 10.05.2017 um 12:18 schrieb Mark Thomas:

On 10/05/17 10:26, Michael Heinen wrote:

Am 10.05.2017 um 00:40 schrieb Mark Thomas:

On 09/05/17 15:25, Michael Heinen wrote:

Hi all,

I am currently mirgating an application from Tomcat 7.0.73 to 8.0.43.
On development platforms we use an embedded tomcat.
On of the jars on the classpath contains a web-fragment.xml in it's
META-INF folder.
This jar file is NOT located within the webApp but simply on the
classpath.
With Tomcat 7.0.73 the web-fragment is processed and the defined
filter
is instantiated.
With Tomcat 8.0.43 the web-fragment is not processed at all.




Should I open a bug?

Maybe. There might be a (spec related) reason for that. E.g. Something
that was tightened up / clarified in Servlet 3.1 that wasn't back-ported
to Tomcat 7 in case users were relying on the functionality.

The next thing to check is the svn history for those files.
https://svn.apache.org/viewvc/tomcat/ and the annotate (a.k.a blame)
view is useful for that. The key is likely to be the commit that added
the isWebapp test.

Mark

I digged into the history an found following changeset with the comment
"Container provided JARs should not be scanned for deployment
annotations nor should they be checked for web-fragment.xml files."
https://svn.apache.org/viewvc?view=revision=1493066
I have no idea how to find the corresponding ticket but a guy called
"markt" implemented this ;-)

If there was an associated Bugzilla issue, a link to it would normally
be included in the commit message.


So the check has been added on purpose but I don't know the reason for
that.
What next?

Let me look at the specs and see if there was a change/clarification
that might have prompted this. I do recall in that timeframe that there
were some clarifications about what got scanned in what circumstances.

/me goes to read spec docs...

The relevant language is in section 8.2.1

only JAR files
bundled in a web application's WEB-INF/lib directory, but not those
higher up in the class loading delegation chain, need to be scanned for
web-fragment.xml


Therefore, Tomcat 8.0 looks to be doing the right thing.

Digging into the archives, finds this thread:
http://tomcat.markmail.org/thread/z3n2lfaa6zlpvodl

It looks like this was missed when we implemented Servlet 3.0 (Tomcat 7)
but was picked up during the 3.1 work.

I haven't looked at the changes in detail but it looks like some
combination of needing the new scanning API to implement it and not
wanting to change 7.0.x behaviour after several years of stable releases
was responsible for it not being back-ported.

Neither web-fragment.xml change nor the wider JarScanner changes are
listed in the 7.0.x to 8.0.x migration guide. That looks like it should
be corrected.

Mark


Thanks Mark for your quick answers!

Michael


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Embedded tomcat does not find web-fragment in jars outside web-inf\lib

[Mark Thomas]

On 10/05/17 12:40, Michael Heinen wrote:
> Am 10.05.2017 um 12:18 schrieb Mark Thomas:
>> On 10/05/17 10:26, Michael Heinen wrote:
>>> Am 10.05.2017 um 00:40 schrieb Mark Thomas:
 On 09/05/17 15:25, Michael Heinen wrote:
> Hi all,
>
> I am currently mirgating an application from Tomcat 7.0.73 to 8.0.43.
> On development platforms we use an embedded tomcat.
> On of the jars on the classpath contains a web-fragment.xml in it's
> META-INF folder.
> This jar file is NOT located within the webApp but simply on the
> classpath.
> With Tomcat 7.0.73 the web-fragment is processed and the defined
> filter
> is instantiated.
> With Tomcat 8.0.43 the web-fragment is not processed at all.



>>> Should I open a bug?
>> Maybe. There might be a (spec related) reason for that. E.g. Something
>> that was tightened up / clarified in Servlet 3.1 that wasn't back-ported
>> to Tomcat 7 in case users were relying on the functionality.
>>
>> The next thing to check is the svn history for those files.
>> https://svn.apache.org/viewvc/tomcat/ and the annotate (a.k.a blame)
>> view is useful for that. The key is likely to be the commit that added
>> the isWebapp test.
>>
>> Mark
> 
> I digged into the history an found following changeset with the comment
> "Container provided JARs should not be scanned for deployment
> annotations nor should they be checked for web-fragment.xml files."
> https://svn.apache.org/viewvc?view=revision=1493066
> I have no idea how to find the corresponding ticket but a guy called
> "markt" implemented this ;-)

If there was an associated Bugzilla issue, a link to it would normally
be included in the commit message.

> So the check has been added on purpose but I don't know the reason for
> that.
> What next?

Let me look at the specs and see if there was a change/clarification
that might have prompted this. I do recall in that timeframe that there
were some clarifications about what got scanned in what circumstances.

/me goes to read spec docs...

The relevant language is in section 8.2.1

only JAR files
bundled in a web application's WEB-INF/lib directory, but not those
higher up in the class loading delegation chain, need to be scanned for
web-fragment.xml


Therefore, Tomcat 8.0 looks to be doing the right thing.

Digging into the archives, finds this thread:
http://tomcat.markmail.org/thread/z3n2lfaa6zlpvodl

It looks like this was missed when we implemented Servlet 3.0 (Tomcat 7)
but was picked up during the 3.1 work.

I haven't looked at the changes in detail but it looks like some
combination of needing the new scanning API to implement it and not
wanting to change 7.0.x behaviour after several years of stable releases
was responsible for it not being back-ported.

Neither web-fragment.xml change nor the wider JarScanner changes are
listed in the 7.0.x to 8.0.x migration guide. That looks like it should
be corrected.

Mark


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Embedded tomcat does not find web-fragment in jars outside web-inf\lib

[Michael Heinen]

Am 10.05.2017 um 12:18 schrieb Mark Thomas:

On 10/05/17 10:26, Michael Heinen wrote:

Am 10.05.2017 um 00:40 schrieb Mark Thomas:

On 09/05/17 15:25, Michael Heinen wrote:

Hi all,

I am currently mirgating an application from Tomcat 7.0.73 to 8.0.43.
On development platforms we use an embedded tomcat.
On of the jars on the classpath contains a web-fragment.xml in it's
META-INF folder.
This jar file is NOT located within the webApp but simply on the
classpath.
With Tomcat 7.0.73 the web-fragment is processed and the defined filter
is instantiated.
With Tomcat 8.0.43 the web-fragment is not processed at all.

Some lines of code:

final StandardContext ctx = ...
ctx.setXmlBlockExternal(false);

final StandardJarScanner jarScanner = new StandardJarScanner();
jarScanner.setScanAllDirectories(true);
jarScanner.setScanAllFiles(true);
jarScanner.setScanBootstrapClassPath(true);
jarScanner.setScanClassPath(true);
jarScanner.setScanManifest(false);
ctx.setJarScanner(jarScanner);

Should this work with Tomcat 8 or is my expectation wrong?

It should work. I'd recommend putting a break-point in
StandardJarScanner and stepping through the Tomcat code to see what is
going wrong.

Mark

Thanks Mark
Here are the results from debugging:

1) Tomcat 8.0.43:
The boolean parameter "isWebapp" is false in this case and therefore the
web-fragment is not processed at all.
See
org.apache.tomcat.util.descriptor.web.FragmentJarScannerCallback.scan(Jar,
String, boolean)

 // Only web application JARs are checked for web-fragment.xml
 // files.
 // web-fragment.xml files don't need to be parsed if they are never
 // going to be used.
 if (isWebapp && parseRequired) {
 is = jar.getInputStream(FRAGMENT_LOCATION);
 }

2) Tomcat 7.0.73:
There is no check for webApps. The member "parseRequired" is true and
therefore the fragment is loaded.
See
org.apache.catalina.startup.ContextConfig.FragmentJarScannerCallback.scan(JarURLConnection)


  if (parseRequired || context.getXmlValidation()) {
 is = jar.getInputStream(FRAGMENT_LOCATION);
  }

Thanks for digging in to that. That certainly explains the different
behaviours.


Should I open a bug?

Maybe. There might be a (spec related) reason for that. E.g. Something
that was tightened up / clarified in Servlet 3.1 that wasn't back-ported
to Tomcat 7 in case users were relying on the functionality.

The next thing to check is the svn history for those files.
https://svn.apache.org/viewvc/tomcat/ and the annotate (a.k.a blame)
view is useful for that. The key is likely to be the commit that added
the isWebapp test.

Mark


I digged into the history an found following changeset with the comment
"Container provided JARs should not be scanned for deployment 
annotations nor should they be checked for web-fragment.xml files."

https://svn.apache.org/viewvc?view=revision=1493066
I have no idea how to find the corresponding ticket but a guy called 
"markt" implemented this ;-)


So the check has been added on purpose but I don't know the reason for 
that.

What next?

Thanks,
Michael




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org





-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Embedded tomcat does not find web-fragment in jars outside web-inf\lib

[Mark Thomas]

On 10/05/17 10:26, Michael Heinen wrote:
> Am 10.05.2017 um 00:40 schrieb Mark Thomas:
>> On 09/05/17 15:25, Michael Heinen wrote:
>>> Hi all,
>>>
>>> I am currently mirgating an application from Tomcat 7.0.73 to 8.0.43.
>>> On development platforms we use an embedded tomcat.
>>> On of the jars on the classpath contains a web-fragment.xml in it's
>>> META-INF folder.
>>> This jar file is NOT located within the webApp but simply on the
>>> classpath.
>>> With Tomcat 7.0.73 the web-fragment is processed and the defined filter
>>> is instantiated.
>>> With Tomcat 8.0.43 the web-fragment is not processed at all.
>>>
>>> Some lines of code:
>>>
>>> final StandardContext ctx = ...
>>> ctx.setXmlBlockExternal(false);
>>>
>>> final StandardJarScanner jarScanner = new StandardJarScanner();
>>> jarScanner.setScanAllDirectories(true);
>>> jarScanner.setScanAllFiles(true);
>>> jarScanner.setScanBootstrapClassPath(true);
>>> jarScanner.setScanClassPath(true);
>>> jarScanner.setScanManifest(false);
>>> ctx.setJarScanner(jarScanner);
>>>
>>> Should this work with Tomcat 8 or is my expectation wrong?
>> It should work. I'd recommend putting a break-point in
>> StandardJarScanner and stepping through the Tomcat code to see what is
>> going wrong.
>>
>> Mark
> 
> Thanks Mark
> Here are the results from debugging:
> 
> 1) Tomcat 8.0.43:
> The boolean parameter "isWebapp" is false in this case and therefore the
> web-fragment is not processed at all.
> See
> org.apache.tomcat.util.descriptor.web.FragmentJarScannerCallback.scan(Jar,
> String, boolean)
> 
> // Only web application JARs are checked for web-fragment.xml
> // files.
> // web-fragment.xml files don't need to be parsed if they are never
> // going to be used.
> if (isWebapp && parseRequired) {
> is = jar.getInputStream(FRAGMENT_LOCATION);
> }
> 
> 2) Tomcat 7.0.73:
> There is no check for webApps. The member "parseRequired" is true and
> therefore the fragment is loaded.
> See
> org.apache.catalina.startup.ContextConfig.FragmentJarScannerCallback.scan(JarURLConnection)
> 
> 
>  if (parseRequired || context.getXmlValidation()) {
> is = jar.getInputStream(FRAGMENT_LOCATION);
>  }

Thanks for digging in to that. That certainly explains the different
behaviours.

> Should I open a bug?

Maybe. There might be a (spec related) reason for that. E.g. Something
that was tightened up / clarified in Servlet 3.1 that wasn't back-ported
to Tomcat 7 in case users were relying on the functionality.

The next thing to check is the svn history for those files.
https://svn.apache.org/viewvc/tomcat/ and the annotate (a.k.a blame)
view is useful for that. The key is likely to be the commit that added
the isWebapp test.

Mark


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Embedded tomcat does not find web-fragment in jars outside web-inf\lib

[Michael Heinen]

Am 10.05.2017 um 00:40 schrieb Mark Thomas:

On 09/05/17 15:25, Michael Heinen wrote:

Hi all,

I am currently mirgating an application from Tomcat 7.0.73 to 8.0.43.
On development platforms we use an embedded tomcat.
On of the jars on the classpath contains a web-fragment.xml in it's
META-INF folder.
This jar file is NOT located within the webApp but simply on the classpath.
With Tomcat 7.0.73 the web-fragment is processed and the defined filter
is instantiated.
With Tomcat 8.0.43 the web-fragment is not processed at all.

Some lines of code:

final StandardContext ctx = ...
ctx.setXmlBlockExternal(false);

final StandardJarScanner jarScanner = new StandardJarScanner();
jarScanner.setScanAllDirectories(true);
jarScanner.setScanAllFiles(true);
jarScanner.setScanBootstrapClassPath(true);
jarScanner.setScanClassPath(true);
jarScanner.setScanManifest(false);
ctx.setJarScanner(jarScanner);

Should this work with Tomcat 8 or is my expectation wrong?

It should work. I'd recommend putting a break-point in
StandardJarScanner and stepping through the Tomcat code to see what is
going wrong.

Mark


Thanks Mark
Here are the results from debugging:

1) Tomcat 8.0.43:
The boolean parameter "isWebapp" is false in this case and therefore the 
web-fragment is not processed at all.
See 
org.apache.tomcat.util.descriptor.web.FragmentJarScannerCallback.scan(Jar, 
String, boolean)


// Only web application JARs are checked for web-fragment.xml
// files.
// web-fragment.xml files don't need to be parsed if they are never
// going to be used.
if (isWebapp && parseRequired) {
is = jar.getInputStream(FRAGMENT_LOCATION);
}

2) Tomcat 7.0.73:
There is no check for webApps. The member "parseRequired" is true and 
therefore the fragment is loaded.
See 
org.apache.catalina.startup.ContextConfig.FragmentJarScannerCallback.scan(JarURLConnection)


 if (parseRequired || context.getXmlValidation()) {
is = jar.getInputStream(FRAGMENT_LOCATION);
 }

Should I open a bug?

Regards,
Michael




Note that the web-fragement is processed when I move the jar to the
web-inf\lib folder.

Regards,

Michael




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org





-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Embedded tomcat does not find web-fragment in jars outside web-inf\lib

[Mark Thomas]

On 09/05/17 15:25, Michael Heinen wrote:
> Hi all,
> 
> I am currently mirgating an application from Tomcat 7.0.73 to 8.0.43.
> On development platforms we use an embedded tomcat.
> On of the jars on the classpath contains a web-fragment.xml in it's
> META-INF folder.
> This jar file is NOT located within the webApp but simply on the classpath.
> With Tomcat 7.0.73 the web-fragment is processed and the defined filter
> is instantiated.
> With Tomcat 8.0.43 the web-fragment is not processed at all.
> 
> Some lines of code:
> 
> final StandardContext ctx = ...
> ctx.setXmlBlockExternal(false);
> 
> final StandardJarScanner jarScanner = new StandardJarScanner();
> jarScanner.setScanAllDirectories(true);
> jarScanner.setScanAllFiles(true);
> jarScanner.setScanBootstrapClassPath(true);
> jarScanner.setScanClassPath(true);
> jarScanner.setScanManifest(false);
> ctx.setJarScanner(jarScanner);
> 
> Should this work with Tomcat 8 or is my expectation wrong?

It should work. I'd recommend putting a break-point in
StandardJarScanner and stepping through the Tomcat code to see what is
going wrong.

Mark


> 
> Note that the web-fragement is processed when I move the jar to the
> web-inf\lib folder.
> 
> Regards,
> 
> Michael
> 
> 
> 
> 
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
> 


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Embedded tomcat does not find web-fragment in jars outside web-inf\lib

[Michael Heinen]

Hi all,

I am currently mirgating an application from Tomcat 7.0.73 to 8.0.43.
On development platforms we use an embedded tomcat.
On of the jars on the classpath contains a web-fragment.xml in it's 
META-INF folder.

This jar file is NOT located within the webApp but simply on the classpath.
With Tomcat 7.0.73 the web-fragment is processed and the defined filter 
is instantiated.

With Tomcat 8.0.43 the web-fragment is not processed at all.

Some lines of code:

final StandardContext ctx = ...
ctx.setXmlBlockExternal(false);

final StandardJarScanner jarScanner = new StandardJarScanner();
jarScanner.setScanAllDirectories(true);
jarScanner.setScanAllFiles(true);
jarScanner.setScanBootstrapClassPath(true);
jarScanner.setScanClassPath(true);
jarScanner.setScanManifest(false);
ctx.setJarScanner(jarScanner);

Should this work with Tomcat 8 or is my expectation wrong?

Note that the web-fragement is processed when I move the jar to the 
web-inf\lib folder.


Regards,

Michael




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org