Hi,

On Tue, 2024-04-16 at 17:40 +0200, Magnus Ihse Bursie wrote:
> On 2024-04-16 10:28, Volker Simonis wrote:
> 
> > Hi, and sorry, I'm completely new to this discussion.
> > 
> > I just wonder how `static-libs-image` is related to the
> > `graal-builder-image` target which is available in mainline since JDK
> > 11 and builds a static version of all the native libraries excluding
> > the Hotspot ones?

I cannot comment on the static-libs-image part, but graal-builder-image
was my doing[1]. It's a vehicle to build an OpenJDK with static libs in
lib/static which is sufficient to use as a from-source version build of
OpenJDK to build Mandrel[2] or GraalVM, which is convenient.

For graal-builder-image, the result must not include libjvm.a in
lib/static of <images-dir>/graal-builder-jdk.

So yes, the two are related.

Thanks,
Severin

[1] https://bugs.openjdk.org/browse/JDK-8236921
[2] https://github.com/graalvm/mandrel

> That's a very good question. You can also ask how it relates to 
> --enable-static-build from JDK-8136556 which has been in the JDK since 
> 9. :-)
> 
> Or, to put in other words, we already have three different ways to 
> produce static libraries in the JDK, each of them created more or less 
> independently of the other. And now Hermetic Java wants to add a fourth.
> 
> This is not a good situation.
> 
> My intention here is to basically retire static-libs-image, 
> graal-builder-image and --enable-static-build, and replace them with a 
> more general solution that supports all these existing use cases, and
> the new use case from the StaticLink.gmk file that the Hermetic Java 
> project wants to integrate.
> 
> Unfortunately, this requires some delicate reverse-engineering of 
> specifications (i.e. "why the heck did they do it like this???") to 
> understand what the different use cases want to achieve. That is the 
> main driver behind my recent work in clearing out technical debt in the 
> native linking part of the build system.
> 
> /Magnus
> 
> 
> > 
> > @Magnus: will the new, static build you're working on also cover
> > `graal-builder-image` or will it be independent of it? From a quick
> > look it seems that currently both `static-libs-image` and
> > `graal-builder-image` are handled in `make/StaticLibsImage.gmk` but
> > I'm not sure if they use the same build functionality. I just wondered
> > why `graal-builder-image` hasn't been mentioned in this thread but
> > maybe it's something as obvious as that it already is the same
> > functionality anway?
> > 
> > Thank you and best regards,
> > Volker
> > 
> > On Tue, Apr 16, 2024 at 3:02 AM Jiangli Zhou <jiangliz...@google.com> wrote:
> > > Magnus, thanks for the response. Please see comments inlined below.
> > > 
> > > On Fri, Apr 12, 2024 at 4:52 AM Magnus Ihse Bursie
> > > <magnus.ihse.bur...@oracle.com> wrote:
> > > > On 2024-04-02 21:16, Jiangli Zhou wrote:
> > > > 
> > > > Hi Magnus,
> > > > 
> > > > In today's zoom meeting with Alan, Ron, Liam and Chuck, we (briefly) 
> > > > discussed how to move forward contributing the static Java related 
> > > > changes (additional runtime fixes/enhancements on top of the existing 
> > > > static support in JDK) from 
> > > > https://github.com/openjdk/leyden/tree/hermetic-java-runtime to JDK 
> > > > mainline.
> > > > 
> > > > Just a bit more details/context below, which may be useful for others 
> > > > reading this thread.
> > > > 
> > > > The https://github.com/openjdk/leyden/tree/hermetic-java-runtime branch 
> > > > currently contains following for supporting hermetic Java (without the 
> > > > launcher work for runtime support):
> > > > 
> > > > 1. Build change for linking the Java launcher (as bin/javastatic) with 
> > > > JDK/hotspot static libraries (.a), mainly in 
> > > > https://github.com/openjdk/leyden/blob/hermetic-java-runtime/make/StaticLink.gmk.
> > > >  The part for creating the complete sets of static libraries (including 
> > > > libjvm.a) has already been included in the mainline since last year. 
> > > > https://github.com/openjdk/leyden/blob/hermetic-java-runtime/make/StaticLink.gmk
> > > >  is in a very raw state and is intended to demonstrate the capability 
> > > > of building a static Java launcher.
> > > > 
> > > > Indeed. It is nowhere near being able to be integrated.
> > > > 
> > > The main purpose of StaticLink.gmk is to support the static-java-image
> > > make target, which can be used to perform the actual static linking
> > > step using libjvm.a and JDK static libraries. That currently doesn't
> > > exist in the JDK mainline. Creating a "fully" statically linked Java
> > > launcher is the first step (out of many) towards supporting
> > > static/hermetic Java.
> > > 
> > > As part of cleaning/refactoring/integrating for the static linking
> > > step, we want to agree and decide/accept on the following:
> > > 
> > > - Support the "fully" statically linked java launcher for testing and
> > > demoing the capability of static JDK support, e.g.
> > >    - Support running jtreg testing using the "fully" statically linked
> > > Java launcher
> > >    - Set up tests in github workflow to help detect any breaking
> > > changes for static support, e.g. new symbol issues introduced by any
> > > changes. There were some earlier discussions on this with Ron and Alan
> > > during the zoom meetings.
> > > - Which JDK native libraries to be statically linked with the new
> > > launcher target? E.g. StaticLink.gmk currently excludes libjsound.a,
> > > libawt_xawt.a, etc from statically linked with the launcher.
> > > - Do we want more than one statically linked launcher target, based on
> > > the set of linked native libraries?
> > > 
> > > Based on the decisions of the above, the launcher static linking part
> > > would mostly be in a different shape when it's integrated into the
> > > mainline. That's why I referred to StaticLink.gmk as in a "very raw"
> > > state.
> > > 
> > > Here is a high-level view of the state of things for static support:
> > > 
> > > (I)  What we already have in the JDK mainline:
> > > - Able to build a complete set of JDK/VM static libraries using
> > > `static-libs-image` make target (necessary for supporting static JDK)
> > > - Compilation for .o files are done separately for the static
> > > libraries and dynamic library (ok for now)
> > > 
> > > (II) What missing:
> > > - Static linking step as mentioned above
> > > 
> > > (III) What needs to be improved (require cleanups and refactoring, and
> > > you mentioned some of those in your response as well):
> > > - Support building both the static libraries and dynamic libraries
> > > using the same set of .o files, instead of separately compiled .o
> > > files. That helps improve build speed and reduce memory overhead for
> > > building JDK. Your current refactoring work aims to help that.
> > > - Clean up the usages of STATIC_BUILD macro. Most of the usages are in
> > > test code.
> > > - Other runtime fixes/enhancements in the leyden
> > > https://github.com/openjdk/leyden/tree/hermetic-java-runtime branch
> > > 
> > > I think most work mentioned in III has dependencies on II. We need a
> > > workable base to be able to build the "fully" statically linked
> > > launcher for building and testing the work mentioned in III, when
> > > integrating any of those to the JDK mainline. The makefile refactoring
> > > work can be done in parallel but does not need to be completed before
> > > we add the static linking step in JDK mainline.
> > > 
> > > > 2. Additional runtime fixes/enhancements on top of the existing static 
> > > > support in JDK, e.g. support further lookup dynamic native library if 
> > > > the built-in native library cannot be found.
> > > > 
> > > > 3. Some initial (prototype) work on supporting hermetic JDK resource 
> > > > files in the jimage (JDK modules image).
> > > > 
> > > > To move forward, one of the earliest items needed is to add the 
> > > > capability of building the fully statically linked Java launcher in JDK 
> > > > mainline. The other static Java runtime changes can be followed up 
> > > > after the launcher linking part, so they can be built and tested as 
> > > > individual PRs created for the JDK mainline. Magnus, you have expressed 
> > > > interest in helping get the launcher linking part (refactor from 
> > > > https://github.com/openjdk/leyden/blob/hermetic-java-runtime/make/StaticLink.gmk)
> > > >  into JDK mainline. What's your thought on prioritizing the launcher 
> > > > static linking part before other makefile clean ups for static 
> > > > libraries?
> > > > 
> > > > Trust me, my absolute top priority now is working on getting the proper 
> > > > build support needed for Hermetic Java. I can't prioritize it any 
> > > > higher.
> > > Thanks!
> > > 
> > > > I am not sure what you are asking for. We can't just merge 
> > > > StaticLink.gmk from your prototype. And even if we did, what good will 
> > > > it do you?
> > > Please see my comments above.
> > > 
> > > > The problem you are running into is that the build system has not been 
> > > > designed to properly support static linking. There are already 3-4 
> > > > hacks in place to get something sort-of useful out, but they are prone 
> > > > to breaking. I assume that we agree that for Hermetic Java to become a 
> > > > success, we need to have a stable foundation for static builds.
> > > > 
> > > > The core problem of all static linking hacks is that they are not 
> > > > integrated in the right place. They need to be a core part of what 
> > > > NativeCompilation delivers, not something done in a separate file. To 
> > > > put it in other words, StaticLink.gmk from your branch do not need 
> > > > cleanup -- it needs to go away, and the functionality moved to the 
> > > > proper place.
> > > > 
> > > > My approach is that NativeCompilation should support doing either only 
> > > > dynamic linking (as today), or static linking (as today with 
> > > > STATIC_LIBS or STATIC_BUILD), or both. The assumption is that the 
> > > > latter will be default, or at least should be tested by default in GHA. 
> > > > For this to work, we need to compile the source code to .o files only 
> > > > once, and then link these .o files either into a dynamic or a static 
> > > > library (or both).
> > > As of today, the leyden
> > > https://github.com/openjdk/leyden/tree/hermetic-java-runtime branch
> > > can build a "fully" statically linked Java launcher. The issue of
> > > compiling the dynamic and static libraries .o files separately is not
> > > a blocker. It's good to have it resolved at some point of time.
> > > 
> > > > This, in turn, require several changes:
> > > > 
> > > > 1) The linking code needs to be cleaned up, and all technical debt 
> > > > needs to be resolved. This is what I have been doing since I started 
> > > > working on static builds for Hermetic Java. JDK-8329704 (which was 
> > > > integrated yesterday) was the first major milestone of this cleanup. 
> > > > Now, the path were to find a library created by the JDK (static or 
> > > > dynamic) is encapsulated in ResolveLibPath. This is currently a 
> > > > monster, but at least all knowledge is collected in a single location, 
> > > > instead of spread over the code base. Getting this simplified is the 
> > > > next step.
> > > > 
> > > > 2) We need to stop passing the STATIC_BUILD define when compiling. This 
> > > > is partially addressed in your PR, where you have replaced #ifdef 
> > > > STATIC_BUILD with a dynamic lookup. But there is also the problem of 
> > > > JNI/JVMTI entry points. I have been pondering how we can compile the 
> > > > code in a way so we support both dynamic and static name resolution, 
> > > > and I think I have a solution.
> > > > 
> > > > This is unfortunately quite complex, and I have started a discussion 
> > > > with Alan if it is possible to update the JNI spec so that both static 
> > > > and dynamic entry points can have the form "JNI_OnLoad_<library-name>". 
> > > > Ideally, I'd like to see us push for this with as much effort as 
> > > > possible. If we got this in place, static builds would be much easier, 
> > > > and the changes required for Hermetic Java even smaller.
> > > Thumbs up! That seems to be a good direction. Currently in the leyden
> > > branch, it first looks up the unique
> > > JNI_OnLoad<_lib_name>|Agent_OnLoad<_lib_name> etc for built-in
> > > libraries, then search for the dynamic libraries using the
> > > conventional naming when necessary. e.g.:
> > > 
> > > https://github.com/openjdk/leyden/commit/a5c886d2e85a0ff0c3712a5488ae61d8c9d7ba1a
> > > https://github.com/openjdk/leyden/commit/1da8e3240e0bd27366d19f2e7dde386e46015135
> > > 
> > > When spec supports JNI_OnLoad_<library-name> and etc. for dynamic
> > > libraries, we may still need to support the conventional naming
> > > without the <_lib_name> part for existing libraries out there.
> > > 
> > > > And finally, on top of all of this, is the question of widening the 
> > > > platform support. To support linux/gcc with objcopy is trivial, but the 
> > > > question about Windows still remain. I have two possible ways forward, 
> > > > one is to check if there is alternative tooling to use (the prime 
> > > > candidate is the clang-ldd), and the other is to try to "fake" a 
> > > > partial linking by concatenating all source code before compiling. This 
> > > > is not ideal, though, for many reasons, and I am not keen on 
> > > > implementing it, not even for testing. And at this point, I have not 
> > > > had time to investigate any of these options much further, since I have 
> > > > been focusing on 1) above.
> > > > 
> > > > A third option is of course to just say that due to toolchain 
> > > > limitations, static linking is not available on Windows.
> > > Thank you for taking this on! Potentially we could consider taking the
> > > objcopy to localizing hotspot symbols on unix-like platforms, based on
> > > https://github.com/openjdk/jdk/pull/17456 discussions. Additional
> > > testing is still needed to verify the solution.
> > > 
> > > > My recommendation is that you keep on working to resolve the (much more 
> > > > thorny) issues of resource access in Hermetic Java in your branch, 
> > > > where you have a prototype static build that works for you. In the 
> > > > meantime, I will make sure that there will be a functioning, stable and 
> > > > robust way of creating static builds in the mainline, that can be 
> > > > regularly tested and not bit-rot, like the static build hacks that has 
> > > > gone in before.
> > > Most of the JDK resources are now supported as hermetic jimage
> > > (lib/modules) bundled in the
> > > https://github.com/openjdk/leyden/tree/hermetic-java-runtime branch.
> > > The remaining sound.properties, ct.sym and .jfc files can be handled
> > > later. Overally, that part of the work has confirmed the hermetic
> > > jimage bundled solution is robust and helps resolve some of the
> > > difficult start-up sequence issues observed when the hermetic resource
> > > was implemented using JAR file based solution.
> > > 
> > > It might be a good idea to follow up on the static linking discussion
> > > in tomorrow's zoom meeting (hope you'll be able to join tomorrow).
> > > 
> > > Thanks!
> > > 
> > > Jiangli
> > > > /Magnus
> > > > 
> > > > 
> > > > 
> > > > Thanks!
> > > > Jiangli
> > > > 
> > > > On Thu, Feb 15, 2024 at 12:01 PM Jiangli Zhou <jiangliz...@google.com> 
> > > > wrote:
> > > > > On Wed, Feb 14, 2024 at 5:07 PM Jiangli Zhou <jiangliz...@google.com> 
> > > > > wrote:
> > > > > > Hi Magnus,
> > > > > > 
> > > > > > Thanks for looking into this from the build perspective.
> > > > > > 
> > > > > > On Wed, Feb 14, 2024 at 1:00 AM Magnus Ihse Bursie
> > > > > > <magnus.ihse.bur...@oracle.com> wrote:
> > > > > > > First some background for build-dev: I have spent some time 
> > > > > > > looking at
> > > > > > > the build implications of the Hermetic Java effort, which is part 
> > > > > > > of
> > > > > > > Project Leyden. A high-level overview is available here:
> > > > > > > https://cr.openjdk.org/~jiangli/hermetic_java.pdf and the current 
> > > > > > > source
> > > > > > > code is here: 
> > > > > > > https://github.com/openjdk/leyden/tree/hermetic-java-runtime.
> > > > > > Some additional hermetic Java related references that are also 
> > > > > > useful:
> > > > > > 
> > > > > > - https://bugs.openjdk.org/browse/JDK-8303796 is an umbrella bug 
> > > > > > that
> > > > > > links to the issues for resolving static linking issues so far
> > > > > > - https://github.com/openjdk/jdk21/pull/26 is the enhancement for
> > > > > > building the complete set of static libraries in JDK/VM, 
> > > > > > particularly
> > > > > > including libjvm.a
> > > > > > 
> > > > > > > Hermetic Java faces several challenges, but the part that is 
> > > > > > > relevant
> > > > > > > for the build system is the ability to create static libraries. 
> > > > > > > We've
> > > > > > > had this functionality (in three different ways...) for some 
> > > > > > > time, but
> > > > > > > it is rather badly implemented.
> > > > > > > 
> > > > > > > As a result of my investigations, I have a bunch of questions. 
> > > > > > > :-) I
> > > > > > > have gotten some answers in private discussion, but for the sake 
> > > > > > > of
> > > > > > > transparency I will repeat them here, to foster an open dialogue.
> > > > > > > 
> > > > > > > 1. Am I correct in understanding that the ultimate goal of this 
> > > > > > > exercise
> > > > > > > is to be able to have jmods which include static libraries (*.a) 
> > > > > > > of the
> > > > > > > native code which the module uses, and that the user can then run 
> > > > > > > a
> > > > > > > special jlink command to have this linked into a single executable
> > > > > > > binary (which also bundles the *.class files and any additional
> > > > > > > resources needed)?
> > > > > > > 
> > > > > > > 2. If so, is the idea to create special kinds of static jmods, 
> > > > > > > like
> > > > > > > java.base-static.jmod, that contains *.a files instead of lib*.so 
> > > > > > > files?
> > > > > > > Or is the idea that the normal jmod should contain both?
> > > > > > > 
> > > > > > > 3. Linking .o and .a files into an executable is a formidable 
> > > > > > > task. Is
> > > > > > > the intention to have jlink call a system-provided ld, or to 
> > > > > > > bundle ld
> > > > > > > with jlink, or to reimplement this functionality in Java?
> > > > > > I have a similar view as Alan responded in your other email thread.
> > > > > > Things are still in the early stage for the general solution.
> > > > > > 
> > > > > > In the https://github.com/openjdk/leyden/tree/hermetic-java-runtime
> > > > > > branch, when configuring JDK with --with-static-java=yes, the JDK
> > > > > > binary contains the following extra artifacts:
> > > > > > 
> > > > > > - static-libs/*.a: The complete set of JDK/VM static libraries
> > > > > > - jdk/bin/javastatic: A demo Java launcher fully statically linked
> > > > > > with the selected JDK .a libraries (e.g. it currently statically 
> > > > > > link
> > > > > > with the headless) and libjvm.a. It's the standard Java launcher
> > > > > > without additional work for hermetic Java.
> > > > > > 
> > > > > > In our prototype for hermetic Java, we build the hermetic executable
> > > > > > image (a single image) from the following input (see description on
> > > > > > singlejar packaging tool in
> > > > > > https://cr.openjdk.org/~jiangli/hermetic_java.pdf):
> > > > > > 
> > > > > > - A customized launcher (with additional work for hermetic) 
> > > > > > executable
> > > > > > fully statically linked with JDK/VM static libraries (.a files),
> > > > > > application natives and dependencies (e.g. in .a static libraries)
> > > > > > - JDK lib/modules, JDK resource files
> > > > > > - Application classes and resource files
> > > > > > 
> > > > > > Including a JDK library .a into the corresponding .jmod would 
> > > > > > require
> > > > > > extracting the .a for linking with the executable. In some systems
> > > > > > that may cause memory overhead due to the extracted copy of the .a
> > > > > > files. I think we should consider the memory overhead issue.
> > > > > > 
> > > > > > One possibility (as Alan described in his response) is for jlink to
> > > > > > invoke the ld on the build system. jlink could pass the needed JDK
> > > > > > static libraries and libjvm.a (provided as part of the JDK binary) 
> > > > > > to
> > > > > > ld based on the modules required for the application.
> > > > > > 
> > > > > I gave a bit more thoughts on this one. For jlink to trigger ld, it
> > > > > would need to know the complete linker options and inputs. Those
> > > > > include options and inputs related to the application part as well. In
> > > > > some usages, it might be easier to handle native linking separately
> > > > > and pass the linker output, the executable to jlink directly. Maybe we
> > > > > could consider supporting different modes for various usages
> > > > > requirements, from static libraries and native linking point of view:
> > > > > 
> > > > > Mode #1
> > > > > Support .jmod packaged natives static libraries, for both JDK/VM .a
> > > > > and application natives and dependencies. If the inputs to jlink
> > > > > include .jmods, jlink can extract the .a libraries and pass the
> > > > > information to ld to link the executable.
> > > > > 
> > > > > Mode #2
> > > > > Support separate .a as jlink input. Jlink could pass the path
> > > > > information to the .a libraries and other linker options to ld to
> > > > > create the executable.
> > > > > 
> > > > > For both mode #1 and #2, jlink would then use the linker output
> > > > > executable to create the final hermetic image.
> > > > > 
> > > > > Mode #3
> > > > > Support a fully linked executable as a jlink input. When a linked
> > > > > executable is given to jlink, it can process it directly with other
> > > > > JDK data/files to create the final image, without native linking step.
> > > > > 
> > > > > Any other thoughts and considerations?
> > > > > 
> > > > > Best,
> > > > > Jiangli
> > > > > 
> > > > > > > 4. Is the intention is to allow users to create their own jmods 
> > > > > > > with
> > > > > > > static libraries, and have these linked in as well? This seems to 
> > > > > > > be the
> > > > > > > case.
> > > > > > An alternative with less memory overhead could be using application
> > > > > > modular JAR and separate .a as the input for jlink.
> > > > > > 
> > > > > > > If that is so, then there will always be the risk for name
> > > > > > > collisions, and we can only minimize the risk by making sure any 
> > > > > > > global
> > > > > > > names are as unique as possible.
> > > > > > Part of the current effort includes resolving the discovered symbol
> > > > > > collision issues with static linking. Will respond to your other 
> > > > > > email
> > > > > > on the symbol issue separately later.
> > > > > > 
> > > > > > > 5. The original implementation of static builds in the JDK, 
> > > > > > > created for
> > > > > > > the Mobile project, used a configure flag, 
> > > > > > > --enable-static-builds, to
> > > > > > > change the entire behavior of the build system to only produce 
> > > > > > > *.a files
> > > > > > > instead of lib*.so. In contrast, the current system is using a 
> > > > > > > special
> > > > > > > target instead.
> > > > > > I think we would need both configure flag and special target for the
> > > > > > static builds.
> > > > > > 
> > > > > > > In my eyes, this is a much worse solution. Apart from
> > > > > > > the conceptual principle (if the build should generate static or 
> > > > > > > dynamic
> > > > > > > libraries is definitely a property of what a "configuration" 
> > > > > > > means),
> > > > > > > this makes it much harder to implement efficiently, since we 
> > > > > > > cannot make
> > > > > > > changes in NativeCompilation.gmk, where they are needed.
> > > > > > For the potential objcopy work to resolve symbol issues, we can add
> > > > > > that conditionally in NativeCompilation.gmk if STATIC_LIBS is true. 
> > > > > > We
> > > > > > have an internal prototype (not included in
> > > > > > https://github.com/openjdk/leyden/tree/hermetic-java-runtime yet) 
> > > > > > done
> > > > > > by one of colleagues for localizing symbols in libfreetype using
> > > > > > objcopy.
> > > > > > 
> > > > > > > That was not as much a question as a statement. 🙂 But here is the
> > > > > > > question: Do you think it would be reasonable to restore the old
> > > > > > > behavior but with the new methods, so that we don't use special 
> > > > > > > targets,
> > > > > > > but instead tells configure to generate static libraries? I'm 
> > > > > > > thinking
> > > > > > > we should have a flag like "--with-library-type=" that can have 
> > > > > > > values
> > > > > > > "dynamic" (which is default), "static" or "both".
> > > > > > If we want to also build a fully statically linked launcher, maybe
> > > > > > --with-static-java? Being able to configure either dynamic, static 
> > > > > > or
> > > > > > both as you suggested also seems to be a good idea.
> > > > > > 
> > > > > > > I am not sure if "both" are needed, but if we want to bundle both 
> > > > > > > lib*.so and *.a files
> > > > > > > into a single jmod file (see question 2 above), then it 
> > > > > > > definitely is.
> > > > > > > In general, the cost of producing two kinds of libraries are quite
> > > > > > > small, compared to the cost of compiling the source code to 
> > > > > > > object files.
> > > > > > Completely agree. It would be good to avoid recompiling the .o file
> > > > > > for static and dynamic builds. As proposed in
> > > > > > https://bugs.openjdk.org/browse/JDK-8303796:
> > > > > > 
> > > > > > It's beneficial to be able to build both .so and .a from the same 
> > > > > > set
> > > > > > of .o files. That would involve some changes to handle the dynamic 
> > > > > > JDK
> > > > > > and static JDK difference at runtime, instead of relying on the
> > > > > > STATIC_BUILD macro.
> > > > > > 
> > > > > > > Finally, I have looked at how to manipulate symbol visibility. 
> > > > > > > There
> > > > > > > seems many ways forward, so I feel confident that we can find a 
> > > > > > > good
> > > > > > > solution.
> > > > > > > 
> > > > > > > One way forward is to use objcopy to manipulate symbol status
> > > > > > > (global/local). There is an option --localize-symbol in objcopy, 
> > > > > > > that
> > > > > > > has been available in objcopy since at least 2.15, which was 
> > > > > > > released
> > > > > > > 2004, so it should be safe to use. But ideally we should avoid 
> > > > > > > using
> > > > > > > objcopy and do this as part of the linking process. This should be
> > > > > > > possible to do, given that we make changes in 
> > > > > > > NativeCompilation.gmk --
> > > > > > > see question 5 above.
> > > > > > > 
> > > > > > > As a fallback, it is also possible to rename symbols, either 
> > > > > > > piecewise
> > > > > > > or wholesale, using objcopy. There are many ways to do this, using
> > > > > > > --prefix-symbols, --redefine-sym or --redefine-syms (note the -s, 
> > > > > > > this
> > > > > > > takes a file with a list of symbols). Thus we can always 
> > > > > > > introduce a
> > > > > > > "post factum namespace" by renaming symbols.
> > > > > > Renaming or redefining the symbol at build time could cause 
> > > > > > confusions
> > > > > > with debugging. That's a concern raised in
> > > > > > https://github.com/openjdk/jdk/pull/17456 discussions.
> > > > > > 
> > > > > > Additionally, redefining symbols using tools like objcopy may not
> > > > > > handle member names referenced in string literals. For example, in
> > > > > > https://github.com/openjdk/jdk/pull/17456 additional changes are
> > > > > > needed in assembling and SA to reflect the symbol change.
> > > > > > 
> > > > > > > So in the end, I think it will be fully possible to produce .a 
> > > > > > > files
> > > > > > > that only has global symbols for the functions that are part of 
> > > > > > > the API
> > > > > > > exposed by that library, and have all other symbols local, and 
> > > > > > > make this
> > > > > > > is in a way that is consistent with the rest of the build system.
> > > > > > > 
> > > > > > > Finally, a note on Hotspot. Due to debugging reasons, we export
> > > > > > > basically all symbols in hotspot as global. This is not 
> > > > > > > reasonable to do
> > > > > > > for a static build. The effect of not exporting those symbols 
> > > > > > > will be
> > > > > > > that SA will not function to 100%. On the other hand, I have no 
> > > > > > > idea if
> > > > > > > SA works at all with a static build. Have you tested this? Is 
> > > > > > > this part
> > > > > > > of the plan to support, or will it be officially dropped for 
> > > > > > > Hermetic Java?
> > > > > > We have done some testing with jtreg SA related tests for the fully
> > > > > > statically linked `javastatic`.
> > > > > > 
> > > > > > If we use objcopy to localize symbols in hotspot, it's not yet clear
> > > > > > what's the impact on SA. We could do some tests. The other question
> > > > > > that I raised is the supported gcc versions (for partial linking)
> > > > > > related to the solution.
> > > > > > 
> > > > > > Best,
> > > > > > Jiangli
> > > > > > 
> > > > > > > /Magnus
> > > > > > > 
> 


Reply via email to