Mark Thomas wrote:
On 01/08/2012 02:53, andreas palsson wrote:
Good morning.
After upgrading to 7.0.29, one of my larger webapps could no longer be deployed due to
"OutOfMemoryError".
As far as I know, this is related to the new feature of annotation scanning
which was enabled in v7.0.29 (see the stack trace below).
Have anyone else experienced the same issue?
There are multiple issues going on here. There have been reported
previously and are understood. I'll try and explain the full story.
The servlet spec requires (and the expert group (EG) confirmed) that
Tomcat always scans for ServletContainerInitializers (SCIs), even if
metadata-complete is true. Tomcat was changed to do this in 7.0.29.
SCIs can have @HandlesTypes annotations. If one is present, Tomcat must
scan every class and its hierarchy looking for matches to the
HandlesTypes. Tomcat caches the results of this scan to speed up
processing so it doesn't repeatedly scan the same classes. If A & B both
inherit from C, when Tomcat scans A it also scans C and caches the
result so it doesn't have to scan C again when checking B. Tomcat was
caching more than it needed to making the cache excessively large. This
is bug 53535 which has already been fixed and will be in 7.0.30 onwards.
This then raises the question of how to turn of scanning for classes
that match HandlesTypes. While the spec says that JARs excluded using an
absolute-ordering are not scanned for matches to HandlesTypes it was
unclear what is meant to happen when metadata-complete is true since
ordering is ignored in that case.
The EG has recently clarified this and stated that JARs excluded using
an absolute-ordering are not scanned for matches to HandlesTypes
regardless of the setting of metadata-complete. The discussion is not
yet complete and we are waiting for it to conclude and before
implementing the results of that discussion in 7.0.30.
Philosophical note :
Without for a minute questioning the technical competence and intellectual brilliance of
the actors involved, for me the above perfectly illustrates a phenomenon that is common in
the computer software industry (and in lawmaking and government).
Start with something that is reasonably simple and works reasonably well. It is a good
bet that over time - and no doubt with the best intentions in the world - this thing will
be the subject of "amendments" and "additional features" which will gradually transform it
into a monster which no single human person is able to comprehend and dominate fully
anymore. Simultaneously and perversely, the increasing complication of the thing will
auto-generate a whole ecosystem of specialists, whose sole social purpose will be to
comprehend parts of the monster and explain it to lesser humans. And since they derive
their income of doing this, they will have all the right incentives to keep on adding
amendments and features, indirectly allowing their own population to prosper and multiply.
Logically but unfortunately, it rarely seems to happen that all this energy would be
focused in the other direction, which could be described as "simplify and optimise".
Over the last 5 years, at constant price, the sheer speed of computer hardware and
networks must have increased by a factor 10 or more. Yet, it still takes about the same
elapsed time to process the average HTTP request and display its results on a
workstation's desktop, and it still takes about the same elapsed time (and cost) to write
a simple web application. In fact, if one really takes into account the time needed to
absorb the technical material required to be able to write a webapp, and keep current with
this material, it probably takes more people, time and money now than 5 years ago.
Note that in the above, I am only using Tomcat as an example because this is a Tomcat
list, and only using Mark's excellent explanation as a trigger for my rambling.
I am aware that annotations are a Java feature, not a Tomcat feature. And I can imagine
that Java annotations do bring some benefits. But when their implementation in Tomcat
brings along the need of scanning hundreds of files and using significantly more memory
and time to start Tomcat, and when avoiding this extra time and memory usage itself
requires further changes and additional parameters and code, I believe that something is
amiss. Not in Tomcat, but deeper.
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
