thanks a lot for taking the time to reply so quickly.
Jason van Zyl wrote:
On Sun, 2004-02-01 at 21:52, Dalibor Topic wrote:
In my experience (with getting different programs to run on free runtimes), things never work out of the box on all platforms, usually. For example, Maven has an ugly dependency on a sun.* class[1]. Thus Maven in CVS breaks on java runtimes that don't [2] implement sun.* classes. A lot of other java code makes unwarranted assumptions about its runtime [3]. Writing portable java code is probably as hard as writing portable C code.
Well, there are certainly can be a few quirks but a comparison between
free runtimes that are so far behind currently used specifications, or
not even compliant, and JVMs that most people use isn't really relevant
in most cases. The unwarranted assumptions you speak of are things that
are present in non-compliant JVMs.
Compliancy to JDK APIs is a seal of approval given out by Sun for passing the TCK. Since the TCK is not available under a free software or open source license, it's hard for free implementations to claim compatibility with JDK APIs, without risking to get sued anyway. ;) That's why kaffe says quite prominently that it's not Java on it's web page on http://www.kaffe.org. Nevertheless, it runs quite a few java applications quite alright ;)
The unwarrented assumptions (i.e. unportable java code) are breaking applications even under compliant JVMs, as can be seen in a rather famous apache.org project, Ant. Citing from the announcement of Ant's 1.5.4 release:
"(1) With JDK 1.4.2 Sun has changed the entry point for javah, therefore Ant 1.5.3's <javah> task doesn't work on JDK 1.4.2 anymore."
See http://nxnw.org/pipermail/sw-announce/2003-August/000181.html for details.
To me, that's a clear case of unwarranted assumptions: assuming that sun.something.javah.something.else is an entry point to a javah tool implemented in java. The JDK API docs explicitely discourage programmers from using the sun.* packages directly. The JDK tool documentation makes no statement in which language javah is written. It's an assumption that leads to unportable java code, that (obviously) didn't work on some compliant, certified JVMs.
While the free runtimes definitely lack a lot of features found in currently used JDKs, a lot of problems I encountered in getting applications to run on free runtimes was caused by that sort of sloppy programming style. So it can be argued, in my opinion, that getting those applications to run on free runtimes, actually results in useful bug fixes, and better applications.
While I certainly agree with you that [1] can easily be remedied you are the first and probably only person to catch [1]. While it's nice having things like Kaffe and Jaguar they are not products that people reach for first when developing with Java. I would like to use Kaffe but in the few times I've tried it's been a world of hurt. But I will certainly fix [1], that's not really a big deal.
One of the very frequently reported bugs for a while last autumn for Xerces was a hardcoded reference to a sun.* class for character conversion. I took part in the discussion of resolving the bug after stumbling over it while I was making eXist run with kaffe. It took quite a few duplicate bug reports to show Xerces developers that something should be done about it, as various people seem to have been quite keen on running Xerces with alternative runtimes, but were bitten by this bug. Eventually, after a couple of months, it was fixed, afaik. In my opinion, Xerces has gained from free runtime users and developers' participation in the development process, just like these users and developers have gained a working Xerces again for their platforms.
I guess we can both quickly agree that the solution to [1] to use the jakarta commons-codec package is a better solution for a variety of reasons for a jakarta project, than to rely on an undocumented, unspecified class for that functionality.
My most interesting problem with Maven and Kaffe combo so far was the tools.jar reference in the forehead.conf file, as maven-1.0rc1 wouldn't start up with it. Since Sun's tools.jar is distributed under a very restrictive license, Kaffe (and other free runtimes) can't ship it, so that any code which wants to mess with tools.jar is bound to fail.
And there we are again in the 'unwarranted assumptions' case: the JDK docs do say something about a tools.jar file in the 'How Classes are found section', but they fail to name the classes present there, since they are in a sun.* package, i.e. 'forbidden' territory. So tools.jar is rather useless, if one writes his Java programs strictly according to the JDK docs. There is no reason to include it, that can be deduced from the JDK docs. Whatever assumptions Maven makes about tools.jar, these assumptions are hardly backed up by the JDK documentation. Or I failed to find it, in which case I'd appreciate an URL ;)
I'd like to use Maven, and you'd like to use Kaffe, so I'm sure we can find a common ground there, and a good solution.
If I understand you correctly, are you saying that for development purposes Maven should be leveraging platform specific repositories?
Yes, if such repositories are available. Otherwise it becomes quite hard to rebuild maven applications from source to verify their integrity, i.e. that they haven't been tampered with.
No, that's simply not true. How would using an OS repository over Maven's own repository help with this in any way shape or form? All the information necessary to build the project is available in the POM. So all you need to do is get your hands on the POM and you could very easily make a tool to build a project.
Usually OS repositories can be rebuilt from source (if the license permits so). There is also the need to be rebuildable from source in order to apply bug fixes to the source code, or other patches.
I've had the assumption that POM dependencies can only be specified to jars, ejbs and plugins according to http://maven.apache.org/reference/project-descriptor.html#dependencies and not to sources. But I definitely agree that having the dependencies listed in the POM makes building a tool that collects the necessary sources and builds them easier. Thanks for providing that useful way of documenting dependencies!
You may want to be able to rebuild everything from scratch, in order to be able to apply a quick source code patch for a security problem for example, and be able to put it all back together.
So what advantage would a native package manager have over Maven doing this?
He could create the distribution specific patch for an urgent security problem, test it on his native OS, and distribute it straight away for his native OS.
The way I understand the current Maven artifact distribution concept, there is a central repository, that is updated by Maven developers upon request through the Maven JIRA. Since a Maven artifact is supposed to be used by more than one operating system, I'd assume that Maven developers are very careful about patching things, and about creating artefacts that are platform-specific.
Let's say a java library has a native component. Let's say Maven distributes both. After a few months, a security bug is discovered in the native component, that manifests itself only under a very specific (and rare) set of conditions on a nieche platform (let's say playstation2-linux for the sake of the argument). I don't think it should be expected from the Maven developers to maintain C++ libraries for playstation2-linux ;)
A distributor finds a workaround to shut the security hole, by using stricter, but unfortunately much slower argument checking in the java part of the library. Should it go into Maven's repository anyway?
I'm afraid that the 'one distribution fits all' approach doesn't work up to the expectations. But of course, that's highly speculative, and I'd love to see a world of 'write once, run anywhere' java applications all woven together by maven or a native packaging application.
I agree with you 100% that Java applications that are packaged up should surely be available in the form of packages that work on various systems. I mailed the JPackage group long, long ago and gave them a heads up on what was going on in Maven land and there was no interest.
As far as I can tell, there is a lot of interest now between the various java application packaging projects in the Linux world for cooperation between each other. It took a while until free java runtimes like kaffe [4] caught up, and started to become suitable platforms to run current jakarta [5] (or other) applications [6] on.
Ok, but I still don't see why Maven would benefit at all from trying to use an OS specific repository when we have a non-OS specific repository.
Because the code in the non-OS specific repository may not work on the OS due to 'unwarranted assumptions' I cited above, for example. To the contrary, the code in the OS specific repository is supposed to be tested and audited on the OS. A simple example would be fixing sloppy usage of ";" and "\" as separators where an OS distributor could patch those, and get the packages working fine, but the patches alone would not warrant a new release for the developers of the package, so Maven may be stuck with the old, not-working-everywhere version.
Then there is a lot more cooking than JPackage. Naoko and RHUG are RedHat's efforts to use gcj to create very fast native executables and libraries from java applications and libraries.
Costin Manolache here at Apache has played with things like Ant and Tomcat to produce binaries with GCJ and unless things have changed drastically in the recent past there really wasn't much of a difference in speed other than startup time. Not to say that these things aren't interesting but the JVMs people typically use are fast enoug.
From what I can find on the latest benchmarks, gcj seems to come on top of Sun's JDK. But I'd assume that's still application specific.
See http://www.mail-archive.com/[EMAIL PROTECTED]/msg04096.html
While it's obviosly quite clever to leave the production to the project ;), I have some doubts about deployment. Platform specific constraints may exist, that a project is not aware of.
Java developers don't want to care about Platform specific constraints, or non-compliant JVMs generally.
Then they'll produce sloppy code that runs by chance on an implementation that meets their assumptions, and break on others.
Let me give you a nice example of how sloppy programmers create broken code, that fails to work on certified, compliant JVMs: End-Of-Line markers. As everyone knows, different OSes use different End-Of-Line markers: Unix uses \n, Windows uses \r\n, MacOs(X) uses \r. When Apple started shipping java for MacOS, they made sure they got the real thing, from Sun, certified, compliant and all that.
They didn't expect to be bitten by sloppy programmers within Sun: some of the classes that need to do End-Of-Line parsing would 'read-ahead' one character when they encountered \r in order to check if the next character is a \n, and push the character back if it wasn't for later reading. That's all quite nice, and well, but when such a sloppy programmed class tried to read the \n over a socket from an apple computer happyly serving its newline (\r) and waiting for input, a program using that class would hang forever. That was an annoying, and apparently common bug, so that Apple released a TechNote to get Sun to fix it: http://developer.apple.com/technotes/tn/tn1157.html .
Java developers may not want to care about such things, but that only means they'll write bad code waiting to blow up on another compliant VM. Writing good, portable java code is not that different from writing good, portable C code: you have to know what's a safe assumption, and what's not. Just saying "I don't want to hear it" won't make the inherent bugs go away ;)
A small example: Debian is one of the huge Linux distributions. It has a lot of users and a vital developer community. Among other nice things, Debian has a Social Contract [7] and an associated set of software guidelines [8]. Those software guidelines make specific requirements on licensing of software that comes with debian. So some applications may need to be split into free and non-free parts in order to get into debian.
Some Jakarta projects [9], for example, have dependencies on non-free software that make it quite hard to put them in the form in which they are released by their developers into debian. They need to be 'sanitized' first.
Again, these are things that you can suggest projects adhere to but
these are not things you can force projects to do.
You're right. Doing a bit of hacking on a free java runtime doesn't give me any more power to tell other people what to do with their spare time, than hacking on Maven would, i.e. zero Newton. ;) Free runtime developers and java software packagers can try to work out a (mutually beneficial) compromise/cooperation between projects, but there is no way they can pressure projects into 'compliance' with some scheme. If things don't work out, everyone takes a deep breath, and simply moves on to the next thing on their agenda.
This is just a small example of platform-specific adaptations being necessary. Putting the burden for them on the project is not very helpful in the short run, I think.
Again, why are you trying to make Java developers think of platform specific issues. These are in fact the things that we want to avoid.
I doubt you can avoid them, or that avoiding them is a good thing to do, as I tried to show with my examples of nice failures in this mail.
<pathetic>
I could go on for a while, citing folklore tales of failures of Java developers that avoided thinking of platform specific issues. Instead, I'll let the Double-Checked locking idiom serve as a monument to that lack of thinking: http://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedLocking.html
</pathetic>
;)
Seriously, though, you assume that software written in Java is portable, while in my experience, it often enough is not. Sometimes the developers misinterpret the API specs, or don't bother reading them and just decide to cast Objects to what seems like the right class to cast to because it's the class Sun's JDK x.y.z happens to pull out of somewhere.
Could you not leverage the repository to make your platform specific
packages for those that wanted to use platform specific packagers?
Well, there is more to packaging than downloading the binaries ;)
You should also want to be able to verify the integrity of the files, to rebuild the binaries if necessary, to uninstall no longer needed packages, to update packages, etc.
Well, why do assume that we don't want the same thing? We in fact do, but we want something that is not specifically targeted at one particular OS. Having use an OS specific repository might possibly be an option but I would never advocate it being the default. That's just not going to happen.
An option is good enough, I guess. If necessary, it can still be made the default on maven packaged for that OS ;)
I'm glad to see that we want to achieve similar goals, and I hope we can learn from each other's efforts.
Maven is a useful build tool. It's also a very nice site-generation system, and probably a few more things that I haven't caught up reading about yet. OTOH, it's not a replacement for a package management system like rpm, urpmi, or dpkg, as far as I can tell.
That's not something we ever tried to solve. And I'm willing to bet it's something the vast majority of Java developers don't think about because they don't use Java packages bundled up in RPMs or the like.
Which is somewhat sad, as using package management can solve quite a few problems with respect to versioning, etc. But you know that already ;)
I think there is some room for cooperation (and interesting dialogue) between the various java packaging projects and Maven developers. They all try to solve the 'satisfying dependencies' problem, for example.
Yes, but we are trying to do it in non-platform specific way.
Sure. Nevertheless, you too may have to face platform specific aspects, eventually, like slight incompatibilities between Sun's compliant JDK implementations, or different file layouts in compliant JDKs, etc. Others have started out like that, and got there before ;)
Well, one of the things we both can agree upon very easily is that checking in artifacts into CVS is a bad idea. That's like checking in object files.
Yes, but that was generally the practice before Maven came along. No large segment of the Java developer population was using native Java packages are probably still aren't and probably won't in future I would wager.
You have a high probability of being right ;)
But if native packaging efforts suceed in solving the problems they are working on, I'd bet on developers of Unix-ish systems using native package management rather than a non-intergrated solution. Which would turn Maven into another Windows-nieche solution, along with executable installers and so on. But it surely is a comfortable nieche ;)
What benefit is there to general, non-platform specific development in using OS specific repositories of Java packages?
We are obviously coming from different ends of the spectrum but I'm not convinced there is any benefit at all. If Maven had to deal with Linux type packaging systems, which are numerous, and ports like systems on BSD and then Windows and whatever else that would nullify the essential benefit of Maven which is to make cross-platform development easier.
Basically, you let the people who understand the peculiar aspects of their platform look at the code, and fix it *if necessary*, instead of relying on a bunch of guys who may have never heard of the platform to fix their java code to run on it ;)
Note the 'if necessary' bit! In an ideal world it would never be necessary, but (as I've tried to show in this mail), the world is not always very close to the ideal. In those cases, I don't believe that the centralized approach can work very well.
Oh, and thank you very much for providing very interesting food for thought. I hope you enjoy this discussion as much as I do.
cheers, dalibor topic
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
