Re: Is it better for JDK to distribute jmod files independently of JDK itself?
Merging jmod on different platforms sounds like an attractive idea. However, I don't care about it during this time, because I need some architectures whose ports are not in OpenJDK main-line as jlink targets, so I can't get JDK from only one provider. Mike Hearn 于2022年2月9日周三 05:01写道： > Hi Glavo, > > My company is currently working on something you can think of > as jpackage++. Amongst other tasks it invokes jlink to create a JDK > distribution for each targeted operating system. It has JPMS support so > I'll post about it on this list when a release is available for public > download. > > We had the same thoughts as you. JMODs look quite wasteful but as Alan > points out, they are the raw inputs to an optimization pipeline. In theory > jlink can do anything. Jigsaw is a 'finished' project, so there is unlikely > to be any changes to OpenJDK packaging anytime soon. The best way is to > lead by example. Unfortunately that would mean creating a new JDK > distribution and some bandwidth costs, so you might be better off asking > Microsoft, Amazon or some other JDK distribution to try something new here. > Although we can ask for OpenJDK to lead by example by making suggestions, > realistically the relevant people are engaged on other higher impact > projects right now. It's just the way it is. > > You've suggested several bits of JMOD related low hanging fruit. It's > probably best to just write a specification and then implement them! > You/I/we could: > > 1. Take some JDKs and make the JMODs available as separate downloads. > Write a simple spec that defines a URL convention, so given a JDK URL the > platform specific versions and JMODs can be easily located via string > interpolation. Then try and get Amazon/Microsoft/etc to implement the spec. > > 2. Define a convention so the target platform is incorporated into JMOD > file names. Then a single set of JMODs can be used to create linked JVMs > for all supported platforms at once. > > 3. More ambitiously, extend the JMOD format with platform specific > directories and write a classloader library that extracts/loads native > libraries on the fly. Then the JAR format can be replaced with JMODs, such > that a single file represents a truly cross-platform artifact even for > cases like JavaFX or java.base, where native libs/different classes are > required. For bonus points, fork the clever jimage format to a separate > spec so the benefits of jimage's perfect hashing and string dedupe can be > obtained from Maven Central downloads, without requiring OpenJDK to commit > to the format. > > None of these projects are difficult. We've considered doing some of them > already, but there were always higher priorities. The limits of JMOD are in > the end just a disk space/bandwidth waste, so we focus for now on higher > impact developer productivity issues. > > >
Re: Is it better for JDK to distribute jmod files independently of JDK itself?
Thank Alan and Mike for their replies. There are problems in this work that I didn't expect before, but I don't think it is difficult to solve. My current idea is to allow jmod to include such a manifest file: fb9264533c96fd3589d5afa4e9736f7351d938b6ce7e6a142a32345842b85690 bin/java 4e3373188e828751cd64a4bac5ecc08967335cfc72d1a7272eaba35be056b0d7 bin/keytool 57bc8681c5525ff907f0ca6bd59fa20e5ed31c36d96bcac299136b54002acb5e classes/META-INF/services/java.nio.file.spi.FileSystemProvider 6eef3965f7348ef15f81b2b8358be7cd68b3cf0902fb19d4f7f475db27cc3b88 classes/com/sun/crypto/provider/AESCipher$AES128_CBC_NoPadding.class d926525e4ef6fa86c48c2bdda05ccb5f469e56394b9932310eeb145a015a327d classes/com/sun/crypto/provider/AESCipher$AES128_CFB_NoPadding.class c0636edd475a650fbeda67b296f21b3524ba5f8587ea8e7b01d1bf6b67cc6f63 classes/com/sun/crypto/provider/AESCipher$AES128_ECB_NoPadding.class d41123cab33597437eb71ef0ffcbd7545ed50615d8f594afdf672826e4a8abd4 classes/com/sun/crypto/provider/AESCipher$AES128_GCM_NoPadding.class ... f4ecb9abe6831695d478d4395bc82e93b5244c5dbaddc4948ca49e3efd81eae1 classes/sun/util/resources/cldr/CurrencyNames_en.class 319683701a1b539ff93479fd63b102bfd01064f26f481f6665e52a7e747acd45 classes/sun/util/resources/cldr/LocaleNames.class c63870397963307f56a3189e7bc5c18426814f0943e265df5447899c641c081f classes/sun/util/resources/cldr/LocaleNamesProvider.class f0294177b22606daf29663784a06b4fc6942e2bc131568f9144178b95389efa3 classes/sun/util/resources/cldr/LocaleNames_en.class fe71ede9dbca736478356f2e9905a28af45dddc17efedcf2b94e5763e65b89f3 classes/sun/util/resources/cldr/TimeZoneNames.class 7e1b1967a91bd8f3f348e39f0bf2440c0b523afdcb59e28c49949317a698e95e classes/sun/util/resources/cldr/TimeZoneNamesProvider.class 8d3258fa4a145ef87b4084a8dab56934d4b0ef57b4a14630f96fc83f1cabc4b8 classes/sun/util/resources/cldr/TimeZoneNames_en.class 701982a49e37d15a9bef6faf5ccf91284a7bd565167c2af35b9232e43580ddd1 classes/sun/util/spi/CalendarProvider.class a88eeb56c61c0df79c8d000a43de10dba48c77130c8ada1af942c9a8ce29a787 conf/net.properties e9441e51f0d29e25313d33a9005640f11ad93c27f01bc46a34bf02f2a98ffd6b conf/sdp/sdp.conf.template f2a00a1dec3b7a097f0815f338a84717ba1017d5d7aae96d842d2188d67c3250 conf/security/java.policy d10da8fc5bdcb2771024c99cd7a5f8ce84954c21124d27a90a670c32dc1002ee conf/security/java.security 6da0747334b0fea7592fd92614b2bbc8b126535e129b1fee483774d914e98eb5 conf/security/policy/README.txt 758b930a526fc670ab7537f8c26321527050a31f5f42149a2dda623c56a0a1a9 conf/security/policy/limited/default_US_export.policy 2b2627548e61316150d47ffc3e6cad465ca05b3cccd4785eb7d21aa7baa0f441 conf/security/policy/limited/default_local.policy 8c3d7648abcd95a272ce12db870082937f4d7f6878d730d83cb7fbb31eb8b2c9 conf/security/policy/limited/exempt_local.policy 758b930a526fc670ab7537f8c26321527050a31f5f42149a2dda623c56a0a1a9 conf/security/policy/unlimited/default_US_export.policy 8d8a318e6d90dfd7e26612d2b6385aa704f686ca6134c551f8928418d92b851a conf/security/policy/unlimited/default_local.policy 915edc29d63cf2323575c8070d6f0d27d7ff30904aba58ac8ec6571b56d4f48c include/classfile_constants.h 1266aea5b9f5d5db1cb6f8e5c6c43cfa7f80bc4f72d7fe42c6131bb939dc70f4 include/jni.h 56467aef065d402cefbfcff80578ea68568270561a0ee210f8db3154bf7702b1 include/jvmti.h 6ee3e52d24bdb4f4d0312dfbab3d47bd524cbdac5540a4d790cac0620c59b3c8 include/jvmticmlr.h 88cb5c33e306900dd35a78d5a439087123b8e91b0986bb5acb42cc9bd2fcc42e include/linux/jni_md.h a69bce275ba7a3570af6579cb0f55682cd75fedfcd49e0e8e9022270c447c916 legal/ADDITIONAL_LICENSE_INFO a44eb7b5caf5534c6ef536b21edb40b4d6babf91bf97d9d45596868618b2c6fb legal/ASSEMBLY_EXCEPTION 4b9abebc4338048a7c2dc184e9f800deb349366bdf28eb23c2677a77b4c87726 legal/LICENSE 45c6d4da48325edfbff3dcf71c704e504c057904435ed23c6d57046d551eb69d legal/aes.md 1ff912740e84e024711def5fa482ffbb46eff64559760c467352dfa7c39a3307 legal/asm.md bef40679922d6fdfb7e4ddb223ad6722300f6054ba737bbf6188d60fcec517f9 legal/c-libutl.md 4491d38ccd79fcda6d14871a3551bced8db26ab2f2232b9d9db2e1eabae25d2f legal/cldr.md af0057e8553906083f69c2fb9fe9ed4ae8bc2340a0b1e376a424702f00300b29 legal/icu.md d9ed58c3132c2c8e82b095eb4ce24cafd1f20531c16a7c9d01f2134843904db7 legal/public_suffix.md 5e4e6623a21a63f9bc16ea54af4133b8038e490c0d499a74676f9e5a61b9c5b2 legal/unicode.md 97551969e2a0f0ebfad1a56f2d6de68a1550790f4004c0fe1bb38ff9c9531bab lib/classlist a54aef23cf04b363c8d960f52221ef5615bc324c94cbe0731459fd46012092e8 lib/jexec 4b90aa83dcfa5dda3088b3305a8d6eb10665ed0e20b556961171503d0e5bb2f6 lib/jrt-fs.jar 3a1a2ec8b5bc95a2b90ff0e997a6267505d8a5959bd5eea39e1a95d3fc2c1553 lib/jspawnhelper aa9efb969444c1484e29adecab55a122458090616e766b2f1230ef05bc3867e0 lib/jvm.cfg 70fd9b3b282196212ad376706b3a994034cf045eeb5c8bab1d4c39b71c6b93df lib/libjava.so 549db6b79b5c0da247cead07e4f1c6186ef13f143fda7d59a727edcf8004d9d9 lib/libjimage.so 00940eec1c6c3aefab64a925bbe350c6da6741c8b659cb871af519450665fd1d lib/libjli.so
Re: Is it better for JDK to distribute jmod files independently of JDK itself?
Hi Glavo, My company is currently working on something you can think of as jpackage++. Amongst other tasks it invokes jlink to create a JDK distribution for each targeted operating system. It has JPMS support so I'll post about it on this list when a release is available for public download. We had the same thoughts as you. JMODs look quite wasteful but as Alan points out, they are the raw inputs to an optimization pipeline. In theory jlink can do anything. Jigsaw is a 'finished' project, so there is unlikely to be any changes to OpenJDK packaging anytime soon. The best way is to lead by example. Unfortunately that would mean creating a new JDK distribution and some bandwidth costs, so you might be better off asking Microsoft, Amazon or some other JDK distribution to try something new here. Although we can ask for OpenJDK to lead by example by making suggestions, realistically the relevant people are engaged on other higher impact projects right now. It's just the way it is. You've suggested several bits of JMOD related low hanging fruit. It's probably best to just write a specification and then implement them! You/I/we could: 1. Take some JDKs and make the JMODs available as separate downloads. Write a simple spec that defines a URL convention, so given a JDK URL the platform specific versions and JMODs can be easily located via string interpolation. Then try and get Amazon/Microsoft/etc to implement the spec. 2. Define a convention so the target platform is incorporated into JMOD file names. Then a single set of JMODs can be used to create linked JVMs for all supported platforms at once. 3. More ambitiously, extend the JMOD format with platform specific directories and write a classloader library that extracts/loads native libraries on the fly. Then the JAR format can be replaced with JMODs, such that a single file represents a truly cross-platform artifact even for cases like JavaFX or java.base, where native libs/different classes are required. For bonus points, fork the clever jimage format to a separate spec so the benefits of jimage's perfect hashing and string dedupe can be obtained from Maven Central downloads, without requiring OpenJDK to commit to the format. None of these projects are difficult. We've considered doing some of them already, but there were always higher priorities. The limits of JMOD are in the end just a disk space/bandwidth waste, so we focus for now on higher impact developer productivity issues.
Re: Is it better for JDK to distribute jmod files independently of JDK itself?
On 08/02/2022 14:17, Glavo wrote: : In addition, I think there are other better options for the out-of-the-box availability of jlink. I noticed that a copy of everything in jmod already exists in the JDK folder. In fact, jmod of JDK core module only needs to store a file list (List of paths of all native libraries, executable files and legal files), original content backup of some modifiable content (e.g. conf files), and the name of module. We can re extract the complete jmod file from JDK through these meta information, the size of these meta information is completely negligible. I'm spending some free time trying to implement such a tool, and I don't think it's difficult. Just an FYI that there are transformations and some class generation at link time that means the classes/resources in the run-time image aren't identical to the those in the packaged modules. -Alan
Is it better for JDK to distribute jmod files independently of JDK itself?
Since Java 9, JDK has been distributed with jmod files bundled with all modules. I am confused by this release strategy. One important problem is that this expands the size of JDK's compressed package by 60% (~ 70MB), and the decompressed size expands by 30%. This takes up a lot of extra hard disk space, and more importantly, it causes a lot of bandwidth waste. Many times we only want a JDK and don't need jlink (especially CI environment), but unfortunately I haven't seen any openjdk distribution provide such things. Conversely, when I use jlink, it also annoys me. I want to use a dedicated Linux machine to compile the program and use jlink to generate the runtime for each platform. On this machine, I only need a JDK of Linux AMD64 and jmod files of JDK of various platforms, but I have never seen any provider provide JDK's jmod files separately, so I have to download one complete JDK after another, even if I can't use them at all except jmod files. When the number of target platforms increases, the waste caused by this increases sharply. These wastes seem to be less important for a single JDK, but in some scenarios (CI, container, virtual machine, etc), a large number of jdks may need to be downloaded and stored, and these wastes cannot be ignored. Now I'm confused about the current distribution method. Would it be a better choice to distribute JDK and jmod separately? In addition, I think there are other better options for the out-of-the-box availability of jlink. I noticed that a copy of everything in jmod already exists in the JDK folder. In fact, jmod of JDK core module only needs to store a file list (List of paths of all native libraries, executable files and legal files), original content backup of some modifiable content (e.g. conf files), and the name of module. We can re extract the complete jmod file from JDK through these meta information, the size of these meta information is completely negligible. I'm spending some free time trying to implement such a tool, and I don't think it's difficult. I think if jlink can recognize this kind of meta information, JDK only needs to bundle these trivial meta information contents, and the complete jmod file is distributed separately from JDK, which may be the best choice. These are my immature ideas. I hope I can get your advice. Thank you.
