Peering into the pit of jar hell - the mess of tomcat's and other jars in RPM distributions
Sorry to open up with venting, but I truly cannot believe how big of a mess that I found of Tomcat's and others' jars under /usr/share/java in a CentOS 5.2 distribution I examined this morning. For years I've been using tar.gz'd Tomcat that I downloaded and applications I used that had standalone installs would provide similar looking directory layouts of Tomcat. All of those were just great. In the RPM'd Tomcat though, the directories are spread out all over the place (which is acceptable), but from what I've been told, the backporting of patches and possibly attempts to lower the number of the same files (jar files in this case) leave you with a ton of sometimes insane looking symlinks and files. Here is what I'm talking about in /usr/share/java if you're unfamiliar: libgcj-4.1.1.jar libgcj-4.1.2.jar - libgcj-4.1.1.jar libgcj-tools-4.1.1.jar libgcj-tools-4.1.2.jar - libgcj-tools-4.1.1.jar Regardless of how trivial a small change in a version of a jar might be in one case for one version of an application, since this is a shared area for jars, you don't know what some other application would expect out of that jar. And if the person trying to track down an issue thinks they are using one version of a jar, but it is really pointed at a different version. xerces-j2.jar - xerces-j2-2.7.1.jar This seems wrong because you can't assume that, just because you are dependent on a certain jar in one application, the same one would apply to multiple applications. One app might be built with 2.7.1 and another with 2.7.3 that didn't deprecate some method that it removed or changed the signature of, and you might not notice that unless every facet of the jar were tested, and if RedHat or the Fedora community has enough time to do that, they certainly aren't spending their time very wisely. wsdl4j.jar - qname-1.5.2.jar This just looks completely wrong, even if they completely merged the same version of the previous jar into the new one: and servletapi5.jar - tomcat5-servlet-2.4-api-5.5.23.jar This seems wrong on a new counts here as it is a specific implementation and specific version paired with a generic jar name symlink. this one is more excusable than the others though. I don't fundamentally disagree with things often, but I really don't agree that this is a good idea, and it is unfortunately one of the worst messes of jars/dependencies I've seen in my last 10 years as a Java developer (and I've seen some pretty messed up things). How and why could someone RPM Tomcat at all if this is the mess that it falls into? TIA for any comments, facts, or opinions you can provide, Gary Please note that I also just wrote a quick entry about this here (feel free to comment there if you'd rather, although they shut it down for comments after a while to avoid blog link spamming): http://weblogs.java.net/blog/garysweaver/archive/2009/05/peering_into_th.html - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Peering into the pit of jar hell - the mess of tomcat's and other jars in RPM distributions
On Fri, May 22, 2009 at 12:01 PM, Gary Weaver gary.wea...@duke.edu wrote: Sorry to open up with venting, but I truly cannot believe how big of a mess that I found of Tomcat's and others' jars under /usr/share/java in a CentOS 5.2 distribution I examined this morning. Uh, shouldn't you be bringing this up on a CentOS mailing list then? :-) -- Hassan Schroeder hassan.schroe...@gmail.com - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Peering into the pit of jar hell - the mess of tomcat's and other jars in RPM distributions
Probably the most common question asked on this list is: Are you using one of those @#$%$#$ Tomcats from a third party distribution? The follow-up is always: You'll have to get help from the people creating that distribution. BTW: On my own CentOS box, I simply ignore the distribution, and use the download from Apache. Cheers, Ken Bowen On May 22, 2009, at 3:01 PM, Gary Weaver wrote: Sorry to open up with venting, but I truly cannot believe how big of a mess that I found of Tomcat's and others' jars under /usr/share/ java in a CentOS 5.2 distribution I examined this morning. For years I've been using tar.gz'd Tomcat that I downloaded and applications I used that had standalone installs would provide similar looking directory layouts of Tomcat. All of those were just great. In the RPM'd Tomcat though, the directories are spread out all over the place (which is acceptable), but from what I've been told, the backporting of patches and possibly attempts to lower the number of the same files (jar files in this case) leave you with a ton of sometimes insane looking symlinks and files. Here is what I'm talking about in /usr/share/java if you're unfamiliar: libgcj-4.1.1.jar libgcj-4.1.2.jar - libgcj-4.1.1.jar libgcj-tools-4.1.1.jar libgcj-tools-4.1.2.jar - libgcj-tools-4.1.1.jar Regardless of how trivial a small change in a version of a jar might be in one case for one version of an application, since this is a shared area for jars, you don't know what some other application would expect out of that jar. And if the person trying to track down an issue thinks they are using one version of a jar, but it is really pointed at a different version. xerces-j2.jar - xerces-j2-2.7.1.jar This seems wrong because you can't assume that, just because you are dependent on a certain jar in one application, the same one would apply to multiple applications. One app might be built with 2.7.1 and another with 2.7.3 that didn't deprecate some method that it removed or changed the signature of, and you might not notice that unless every facet of the jar were tested, and if RedHat or the Fedora community has enough time to do that, they certainly aren't spending their time very wisely. wsdl4j.jar - qname-1.5.2.jar This just looks completely wrong, even if they completely merged the same version of the previous jar into the new one: and servletapi5.jar - tomcat5-servlet-2.4-api-5.5.23.jar This seems wrong on a new counts here as it is a specific implementation and specific version paired with a generic jar name symlink. this one is more excusable than the others though. I don't fundamentally disagree with things often, but I really don't agree that this is a good idea, and it is unfortunately one of the worst messes of jars/dependencies I've seen in my last 10 years as a Java developer (and I've seen some pretty messed up things). How and why could someone RPM Tomcat at all if this is the mess that it falls into? TIA for any comments, facts, or opinions you can provide, Gary Please note that I also just wrote a quick entry about this here (feel free to comment there if you'd rather, although they shut it down for comments after a while to avoid blog link spamming): http://weblogs.java.net/blog/garysweaver/archive/2009/05/peering_into_th.html - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Peering into the pit of jar hell - the mess of tomcat's and other jars in RPM distributions
From: Gary Weaver [mailto:gary.wea...@duke.edu] Subject: Peering into the pit of jar hell - the mess of tomcat's and otherjars in RPM distributions For years I've been using tar.gz'd Tomcat that I downloaded and applications I used that had standalone installs would provide similar looking directory layouts of Tomcat. All of those were just great. In the RPM'd Tomcat though, the directories are spread out all over the place (which is acceptable), but from what I've been told, the backporting of patches and possibly attempts to lower the number of the same files (jar files in this case) leave you with a ton of sometimes insane looking symlinks and files. Yup - which is why many of us strongly recommend not to use the 3rd-party repackaged junk, but instead get a real Tomcat download. Unfortunately, the Tomcat committers have no control over what the repackagers do, and the repackagers seem oblivious to the damage done and trouble caused. Here is what I'm talking about in /usr/share/java if you're unfamiliar: libgcj-4.1.1.jar libgcj-4.1.2.jar - libgcj-4.1.1.jar libgcj-tools-4.1.1.jar libgcj-tools-4.1.2.jar - libgcj-tools-4.1.1.jar The above has nothing to do with Tomcat, luckily. BTW, you should toss the libgcj stuff if you need to run serious Java programs, and install a real JVM. I don't fundamentally disagree with things often, but I really don't agree that this is a good idea, and it is unfortunately one of the worst messes of jars/dependencies I've seen in my last 10 years as a Java developer (and I've seen some pretty messed up things). Completely concur - but it's not done by Apache Tomcat, but rather whoever decided to repackage it in an anti-social manner. You can lodge your complaints to CentOS, and then stick with running a real Tomcat (and JVM). - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Peering into the pit of jar hell - the mess of tomcat's and other jars in RPM distributions
MGGood Afternoon Gary
MG(hopefully brief) comment annotations displayed below
Martin
__
Verzicht und Vertraulichkeitanmerkung/Note de déni et de confidentialité
Diese Nachricht ist vertraulich. Sollten Sie nicht der vorgesehene Empfaenger
sein, so bitten wir hoeflich um eine Mitteilung. Jede unbefugte Weiterleitung
oder Fertigung einer Kopie ist unzulaessig. Diese Nachricht dient lediglich dem
Austausch von Informationen und entfaltet keine rechtliche Bindungswirkung.
Aufgrund der leichten Manipulierbarkeit von E-Mails koennen wir keine Haftung
fuer den Inhalt uebernehmen.
Ce message est confidentiel et peut être privilégié. Si vous n'êtes pas le
destinataire prévu, nous te demandons avec bonté que pour satisfaire informez
l'expéditeur. N'importe quelle diffusion non autorisée ou la copie de ceci est
interdite. Ce message sert à l'information seulement et n'aura pas n'importe
quel effet légalement obligatoire. Étant donné que les email peuvent facilement
être sujets à la manipulation, nous ne pouvons accepter aucune responsabilité
pour le contenu fourni.
Date: Fri, 22 May 2009 15:01:37 -0400
From: gary.wea...@duke.edu
To: users@tomcat.apache.org
Subject: Peering into the pit of jar hell - the mess of tomcat's and other
jars in RPM distributions
Sorry to open up with venting, but I truly cannot believe how big of a
mess that I found of Tomcat's and others' jars under /usr/share/java in
a CentOS 5.2 distribution I examined this morning.
For years I've been using tar.gz'd Tomcat that I downloaded and
applications I used that had standalone installs would provide similar
looking directory layouts of Tomcat. All of those were just great.
In the RPM'd Tomcat though, the directories are spread out all over the
place (which is acceptable), but from what I've been told, the
backporting of patches and possibly attempts to lower the number of the
same files (jar files in this case) leave you with a ton of sometimes
insane looking symlinks and files.
Here is what I'm talking about in /usr/share/java if you're unfamiliar:
libgcj-4.1.1.jar
libgcj-4.1.2.jar - libgcj-4.1.1.jar
libgcj-tools-4.1.1.jar
libgcj-tools-4.1.2.jar - libgcj-tools-4.1.1.jar
Regardless of how trivial a small change in a version of a jar might be
in one case for one version of an application, since this is a shared
area for jars, you don't know what some other application would expect
out of that jar. And if the person trying to track down an issue thinks
they are using one version of a jar, but it is really pointed at a
different version.
xerces-j2.jar - xerces-j2-2.7.1.jar
This seems wrong because you can't assume that, just because you are
dependent on a certain jar in one application, the same one would apply
to multiple applications. One app might be built with 2.7.1 and another
with 2.7.3 that didn't deprecate some method that it removed or changed
the signature of, and you might not notice that unless every facet of
the jar were tested, and if RedHat or the Fedora community has enough
time to do that, they certainly aren't spending their time very wisely.
wsdl4j.jar - qname-1.5.2.jar
MGwhat is qname? who builds qname? a quick glance at wsdl4j build.xml
contains:
MGjar jarfile=${build.lib}/qname.jar basedir=${build.dest}
MG include name=javax/xml/**/*.class/
MGwho in wsdl4j references wsdl4j?
MG%WSDL4J_HOME%\src\javaxgrep -S -l wsdl4j *.*
MG .\wsdl\OperationType.java
MGonly 1 reference here and thats a comment
MGthat means qname is now the DRIVER (and wsdl4j is supporting library)
MG/* The following equals method is not used within wsdl4j but
MG * it is historically part of the jsr110 wsdl4j API, so it
MG * will not likely be removed. Although it overloads the
MG * Object.equals method (i.e. it has a different arg) it does
MG * not override it, so Object.equals will still be used by
MG * the readResolve method at deserialization.
MG */
MG
MGwhat is wsdl4j? who builds wsdl4j? a quick glance at wsdl4j build.xml
contains:
MGjar jarfile=${build.lib}/${name}.jar basedir=${build.dest}
MG exclude name=javax/xml/**/
MGa quick hop to the ibm code tree and a grep on qname displays
MG%WSDL4J_HOME%\src\comgrep -S -l qname *.*
MG.\ibm\wsdl\AbstractWSDLElement.java
MG.\ibm\wsdl\DefinitionImpl.java
MG.\ibm\wsdl\extensions\schema\SchemaConstants.java
MG.\ibm\wsdl\util\xml\QNameUtils.java
MG.\ibm\wsdl\xml\WSDLReaderImpl.java
MGas you can see wsdl4j functions are empty stubs that call to qname for real
work
MG(the justification for symlink alias of wsdl4j.jar - qname-1.5.2.jar is
validated)
This just looks completely wrong, even if they completely merged the
same version of the previous jar into the new one:
and
servletapi5.jar - tomcat5-servlet-2.4-api-5.5.23.jar
This seems wrong on a new counts here as it is a specific implementation
and specific version paired with a
Re: Peering into the pit of jar hell - the mess of tomcat's and other jars in RPM distributions
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Gary, On 5/22/2009 3:01 PM, Gary Weaver wrote: Sorry to open up with venting, but I truly cannot believe how big of a mess that I found of Tomcat's and others' jars under /usr/share/java in a CentOS 5.2 distribution I examined this morning. Oh, this is gonna be great ;) Here is what I'm talking about in /usr/share/java if you're unfamiliar: libgcj-4.1.1.jar libgcj-4.1.2.jar - libgcj-4.1.1.jar libgcj-tools-4.1.1.jar libgcj-tools-4.1.2.jar - libgcj-tools-4.1.1.jar Regardless of how trivial a small change in a version of a jar might be in one case for one version of an application, since this is a shared area for jars, you don't know what some other application would expect out of that jar. And if the person trying to track down an issue thinks they are using one version of a jar, but it is really pointed at a different version. xerces-j2.jar - xerces-j2-2.7.1.jar It's even more obnoxious to link the 4.1.2 version to the 4.1.1 JAR since you'd think you were getting 4.1.2 but you're really getting 4.1.1. Shared libraries do this all the time, but they do it in a sane way: libfoo.so - libfoo.1.so libfoo.1.so - libfoo.1.2.3.4.5.so libfoo.1.2.3.4.5.so (this is the real one) So, basically, clients can either load a specific version (1.2.3.4.5) or the latest in a major version (1) and they always get what's appropriate. They can also say latest by loading libfoo.so directly. This is great for shared libraries and /might/ be great for JAR files, but the above does not look sane to me. You wouldn't link libfoo.1.so to libfoo.2.so. :( servletapi5.jar - tomcat5-servlet-2.4-api-5.5.23.jar This seems wrong on a new counts here as it is a specific implementation and specific version paired with a generic jar name symlink. this one is more excusable than the others though. The API JAR should not be stuck to a specific version of Tomcat: this is a generic JAR. I suspect this packaging reflects an ignorance of the relationship between the Servlet API and Tomcat. The servlet 2.4 API is the servlet 2.4 API, no matter who is using it. The version used for 5.5.23 is (or at least should be!) the same as is used for 5.5.1 and 5.5.27. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkoXCBkACgkQ9CaO5/Lv0PDEIgCfU9mbxPF0M0EEuUMUgb5sTobw wvQAn38hwlIF7I2n9LtbEgEGe/EbmUr0 =EUPj -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Peering into the pit of jar hell - the mess of tomcat's and other jars in RPM distributions
Martin, Thanks much for the time you spent on the explanation. However (and hopefully I'm being brief also)- one of issues in doing this is that wsdl4j.jar could (in-general) be any version of wsdl4j not necessarily something that just happens to be populated with one or more classes that do nothing more than have methods that just then call classes in (version specific, because method signatures/classes/packages could change in diff versions) qname-1.5.2.jar. (This is - if that is what you are saying- I do not know what version of wsdl4j it is here, nor have I looked at the source, since I don't know what version it is from the name). The impression I get from looking at the mess of symlinks is that people are assuming (like vendors of Windows products that contributed to Microsoft's DLL hell starting mostly in Win95) that playing around with filenames and versions is perfectly acceptable if it gets the job done (for reducing space they take up in an attempt to share files, or in this case possible reducing the stack level by bypassing methods in an interface jar completely). But in fact, when this is done the only substantial good it does that is not outweighed by negatives is that RedHat will end up selling more support licenses for people that get fed up with RPMs on CentOS/Fedora not working properly (after all, they make money off of support, right?). That maybe a fatalistic viewpoint on my part, and I don't mean to start a firestorm, but basically (in this case) unless you were to have a directory that contained a bunch of jars where each filename were to have a version that actually corresponds to the well-known version of that specific jar, then I think you are really asking for trouble. Thanks, Gary Martin Gainty wrote: MGGood Afternoon Gary MG(hopefully brief) comment annotations displayed below Martin __ Verzicht und Vertraulichkeitanmerkung/Note de déni et de confidentialité Diese Nachricht ist vertraulich. Sollten Sie nicht der vorgesehene Empfaenger sein, so bitten wir hoeflich um eine Mitteilung. Jede unbefugte Weiterleitung oder Fertigung einer Kopie ist unzulaessig. Diese Nachricht dient lediglich dem Austausch von Informationen und entfaltet keine rechtliche Bindungswirkung. Aufgrund der leichten Manipulierbarkeit von E-Mails koennen wir keine Haftung fuer den Inhalt uebernehmen. Ce message est confidentiel et peut être privilégié. Si vous n'êtes pas le destinataire prévu, nous te demandons avec bonté que pour satisfaire informez l'expéditeur. N'importe quelle diffusion non autorisée ou la copie de ceci est interdite. Ce message sert à l'information seulement et n'aura pas n'importe quel effet légalement obligatoire. Étant donné que les email peuvent facilement être sujets à la manipulation, nous ne pouvons accepter aucune responsabilité pour le contenu fourni. Date: Fri, 22 May 2009 15:01:37 -0400 From: gary.wea...@duke.edu To: users@tomcat.apache.org Subject: Peering into the pit of jar hell - the mess of tomcat's and other jars in RPM distributions Sorry to open up with venting, but I truly cannot believe how big of a mess that I found of Tomcat's and others' jars under /usr/share/java in a CentOS 5.2 distribution I examined this morning. For years I've been using tar.gz'd Tomcat that I downloaded and applications I used that had standalone installs would provide similar looking directory layouts of Tomcat. All of those were just great. In the RPM'd Tomcat though, the directories are spread out all over the place (which is acceptable), but from what I've been told, the backporting of patches and possibly attempts to lower the number of the same files (jar files in this case) leave you with a ton of sometimes insane looking symlinks and files. Here is what I'm talking about in /usr/share/java if you're unfamiliar: libgcj-4.1.1.jar libgcj-4.1.2.jar - libgcj-4.1.1.jar libgcj-tools-4.1.1.jar libgcj-tools-4.1.2.jar - libgcj-tools-4.1.1.jar Regardless of how trivial a small change in a version of a jar might be in one case for one version of an application, since this is a shared area for jars, you don't know what some other application would expect out of that jar. And if the person trying to track down an issue thinks they are using one version of a jar, but it is really pointed at a different version. xerces-j2.jar - xerces-j2-2.7.1.jar This seems wrong because you can't assume that, just because you are dependent on a certain jar in one application, the same one would apply to multiple applications. One app might be built with 2.7.1 and another with 2.7.3 that didn't deprecate some method that it removed or changed the signature of, and you might not notice that unless every facet of the jar were tested, and if RedHat or the Fedora community has enough time to do that, they certainly aren't spending their time very wisely. wsdl4j.jar - qname-1.5.2.jar
